Command line
python /home/saxelrod/Repo/projects/chemprop/chemprop/train.py --config_path /home/saxelrod/chemprop_cov_2/models/from_cp_whim_feats_mpnn_from_auc/config.json --data_path /home/saxelrod/rgb_nfs/dsets/cov_2_cl/train_full.csv --dataset_type classification
Args
{'activation': 'ReLU',
 'aggregation': 'mean',
 'aggregation_norm': 100,
 'atom_descriptors': None,
 'atom_descriptors_path': None,
 'atom_descriptors_size': 0,
 'atom_features_size': 0,
 'atom_messages': False,
 'batch_size': 50,
 'bias': False,
 'cache_cutoff': 10000,
 'checkpoint_dir': None,
 'checkpoint_path': None,
 'checkpoint_paths': None,
 'class_balance': True,
 'config_path': '/home/saxelrod/chemprop_cov_2/models/from_cp_whim_feats_mpnn_from_auc/config.json',
 'crossval_index_dir': None,
 'crossval_index_file': None,
 'crossval_index_sets': None,
 'cuda': True,
 'data_path': '/home/saxelrod/rgb_nfs/dsets/cov_2_cl/train_full.csv',
 'dataset_type': 'classification',
 'depth': 5,
 'device': device(type='cuda', index=1),
 'dropout': 0.05,
 'ensemble_size': 1,
 'epochs': 300,
 'extra_metrics': [],
 'features_generator': None,
 'features_only': False,
 'features_path': ['/home/saxelrod/chemprop_cov_2/features/from_cp_whim/train_auc.npz'],
 'features_scaling': False,
 'features_size': None,
 'ffn_hidden_size': 300,
 'ffn_num_layers': 2,
 'final_lr': 0.0001,
 'folds_file': None,
 'gpu': 1,
 'grad_clip': None,
 'hidden_size': 300,
 'ignore_columns': None,
 'init_lr': 0.0001,
 'log_frequency': 10,
 'max_data_size': None,
 'max_lr': 0.001,
 'metric': 'auc',
 'metrics': ['auc'],
 'minimize_score': False,
 'mpn_shared': False,
 'multiclass_num_classes': 3,
 'no_cache_mol': False,
 'no_cuda': False,
 'no_features_scaling': True,
 'num_folds': 10,
 'num_lrs': 1,
 'num_tasks': 1,
 'num_workers': 8,
 'number_of_molecules': 1,
 'pytorch_seed': 0,
 'quiet': True,
 'save_dir': '/home/saxelrod/chemprop_cov_2/models/from_cp_whim_feats_mpnn_from_auc',
 'save_preds': False,
 'save_smiles_splits': False,
 'seed': 0,
 'separate_test_features_path': ['/home/saxelrod/chemprop_cov_2/features/from_cp_whim/test_auc.npz'],
 'separate_test_path': '/home/saxelrod/rgb_nfs/dsets/cov_2_cl/test_full.csv',
 'separate_val_features_path': ['/home/saxelrod/chemprop_cov_2/features/from_cp_whim/val_auc.npz'],
 'separate_val_path': '/home/saxelrod/rgb_nfs/dsets/cov_2_cl/val_full.csv',
 'show_individual_scores': False,
 'smiles_columns': [None],
 'split_sizes': (0.8, 0.1, 0.1),
 'split_type': 'random',
 'target_columns': None,
 'task_names': ['sars_cov_two_cl_protease_active'],
 'test': False,
 'test_fold_index': None,
 'train_data_size': None,
 'undirected': False,
 'use_input_features': True,
 'val_fold_index': None,
 'warmup_epochs': 2.0}
Loading data
Number of tasks = 1
Fold 0
Splitting data with seed 0
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.05, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=300, bias=False)
        (W_h): Linear(in_features=300, out_features=300, bias=False)
        (W_o): Linear(in_features=433, out_features=300, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.05, inplace=False)
    (1): Linear(in_features=2200, out_features=300, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.05, inplace=False)
    (4): Linear(in_features=300, out_features=1, bias=True)
  )
)
Number of parameters = 924,901
Moving model to cuda
Epoch 0
Validation auc = 0.661972
Epoch 1
Validation auc = 0.690610
Epoch 2
Validation auc = 0.694366
Epoch 3
Validation auc = 0.669953
Epoch 4
Loss = 6.9173e-01, PNorm = 37.1929, GNorm = 7.3965, lr_0 = 6.5000e-04
Validation auc = 0.742723
Epoch 5
Validation auc = 0.756338
Epoch 6
Validation auc = 0.695305
Epoch 7
Validation auc = 0.742254
Epoch 8
Validation auc = 0.708920
Epoch 9
Loss = 6.0596e-01, PNorm = 37.2530, GNorm = 8.6332, lr_0 = 9.9743e-04
Validation auc = 0.712207
Epoch 10
Validation auc = 0.689671
Epoch 11
Validation auc = 0.646948
Epoch 12
Validation auc = 0.623005
Epoch 13
Validation auc = 0.619718
Epoch 14
Loss = 5.0149e-01, PNorm = 37.3500, GNorm = 7.4341, lr_0 = 9.8890e-04
Validation auc = 0.630986
Epoch 15
Validation auc = 0.604695
Epoch 16
Validation auc = 0.601408
Epoch 17
Validation auc = 0.624883
Epoch 18
Validation auc = 0.590141
Epoch 19
Loss = 3.8439e-01, PNorm = 37.4611, GNorm = 1.3241, lr_0 = 9.8045e-04
Validation auc = 0.600939
Epoch 20
Validation auc = 0.636150
Epoch 21
Validation auc = 0.635211
Epoch 22
Validation auc = 0.641784
Epoch 23
Validation auc = 0.629577
Epoch 24
Loss = 3.5384e-01, PNorm = 37.5878, GNorm = 1.5181, lr_0 = 9.7207e-04
Validation auc = 0.616432
Epoch 25
Validation auc = 0.600000
Epoch 26
Validation auc = 0.608920
Epoch 27
Validation auc = 0.597653
Epoch 28
Validation auc = 0.613146
Epoch 29
Loss = 3.4547e-01, PNorm = 37.7380, GNorm = 4.8757, lr_0 = 9.6376e-04
Validation auc = 0.620188
Epoch 30
Validation auc = 0.629577
Epoch 31
Validation auc = 0.638028
Epoch 32
Validation auc = 0.641315
Epoch 33
Validation auc = 0.649765
Epoch 34
Loss = 3.5995e-01, PNorm = 37.9070, GNorm = 8.9706, lr_0 = 9.5552e-04
Validation auc = 0.652113
Epoch 35
Validation auc = 0.635681
Epoch 36
Validation auc = 0.634272
Epoch 37
Validation auc = 0.615493
Epoch 38
Validation auc = 0.579343
Epoch 39
Loss = 3.0600e-01, PNorm = 38.0785, GNorm = 2.8011, lr_0 = 9.4735e-04
Validation auc = 0.622535
Epoch 40
Validation auc = 0.622535
Epoch 41
Validation auc = 0.616432
Epoch 42
Validation auc = 0.627230
Epoch 43
Validation auc = 0.628638
Epoch 44
Loss = 2.2283e-01, PNorm = 38.2574, GNorm = 5.3798, lr_0 = 9.3925e-04
Validation auc = 0.607981
Epoch 45
Validation auc = 0.623944
Epoch 46
Validation auc = 0.631925
Epoch 47
Validation auc = 0.629108
Epoch 48
Validation auc = 0.626291
Epoch 49
Loss = 3.6215e-01, PNorm = 38.4487, GNorm = 3.8971, lr_0 = 9.3122e-04
Validation auc = 0.632864
Epoch 50
Validation auc = 0.633803
Epoch 51
Validation auc = 0.636620
Epoch 52
Validation auc = 0.586854
Epoch 53
Validation auc = 0.630986
Epoch 54
Loss = 2.1450e-01, PNorm = 38.6594, GNorm = 1.6285, lr_0 = 9.2326e-04
Validation auc = 0.625352
Epoch 55
Validation auc = 0.613615
Epoch 56
Validation auc = 0.615962
Epoch 57
Validation auc = 0.610329
Epoch 58
Validation auc = 0.604695
Epoch 59
Loss = 1.5167e-01, PNorm = 38.8903, GNorm = 1.4235, lr_0 = 9.1537e-04
Validation auc = 0.624883
Epoch 60
Validation auc = 0.620657
Epoch 61
Validation auc = 0.635211
Epoch 62
Validation auc = 0.634272
Epoch 63
Validation auc = 0.653991
Epoch 64
Loss = 2.5495e-01, PNorm = 39.1038, GNorm = 11.8495, lr_0 = 9.0754e-04
Validation auc = 0.645070
Epoch 65
Validation auc = 0.623944
Epoch 66
Validation auc = 0.614085
Epoch 67
Validation auc = 0.586854
Epoch 68
Validation auc = 0.574178
Epoch 69
Loss = 1.3533e-01, PNorm = 39.2742, GNorm = 1.5909, lr_0 = 8.9978e-04
Validation auc = 0.610798
Epoch 70
Validation auc = 0.610329
Epoch 71
Validation auc = 0.611737
Epoch 72
Validation auc = 0.599061
Epoch 73
Validation auc = 0.614085
Epoch 74
Loss = 1.3263e-01, PNorm = 39.4729, GNorm = 3.0179, lr_0 = 8.9209e-04
Validation auc = 0.618310
Epoch 75
Validation auc = 0.621127
Epoch 76
Validation auc = 0.625352
Epoch 77
Validation auc = 0.629577
Epoch 78
Validation auc = 0.635681
Epoch 79
Loss = 9.4365e-02, PNorm = 39.6598, GNorm = 1.2067, lr_0 = 8.8447e-04
Validation auc = 0.636620
Epoch 80
Validation auc = 0.645540
Epoch 81
Validation auc = 0.646479
Epoch 82
Validation auc = 0.645540
Epoch 83
Validation auc = 0.639906
Epoch 84
Loss = 6.3005e-02, PNorm = 39.8173, GNorm = 0.9584, lr_0 = 8.7691e-04
Validation auc = 0.638498
Epoch 85
Validation auc = 0.640376
Epoch 86
Validation auc = 0.638028
Epoch 87
Validation auc = 0.636620
Epoch 88
Validation auc = 0.631925
Epoch 89
Loss = 8.8516e-02, PNorm = 39.9755, GNorm = 0.9550, lr_0 = 8.6941e-04
Validation auc = 0.642254
Epoch 90
Validation auc = 0.648357
Epoch 91
Validation auc = 0.649296
Epoch 92
Validation auc = 0.647887
Epoch 93
Validation auc = 0.638967
Epoch 94
Loss = 9.9437e-02, PNorm = 40.1190, GNorm = 1.7420, lr_0 = 8.6198e-04
Validation auc = 0.638498
Epoch 95
Validation auc = 0.638028
Epoch 96
Validation auc = 0.633803
Epoch 97
Validation auc = 0.635211
Epoch 98
Validation auc = 0.641315
Epoch 99
Loss = 1.0378e-01, PNorm = 40.2842, GNorm = 1.3756, lr_0 = 8.5461e-04
Validation auc = 0.646948
Epoch 100
Validation auc = 0.644601
Epoch 101
Validation auc = 0.646009
Epoch 102
Validation auc = 0.656338
Epoch 103
Validation auc = 0.667136
Epoch 104
Loss = 6.9544e-02, PNorm = 40.4439, GNorm = 1.1207, lr_0 = 8.4730e-04
Validation auc = 0.632864
Epoch 105
Validation auc = 0.632394
Epoch 106
Validation auc = 0.640845
Epoch 107
Validation auc = 0.625352
Epoch 108
Validation auc = 0.617840
Epoch 109
Loss = 1.0310e-01, PNorm = 40.6117, GNorm = 0.5780, lr_0 = 8.4006e-04
Validation auc = 0.623474
Epoch 110
Validation auc = 0.634742
Epoch 111
Validation auc = 0.647887
Epoch 112
Validation auc = 0.649296
Epoch 113
Validation auc = 0.645070
Epoch 114
Loss = 5.1886e-02, PNorm = 40.7445, GNorm = 1.3682, lr_0 = 8.3288e-04
Validation auc = 0.649296
Epoch 115
Validation auc = 0.640845
Epoch 116
Validation auc = 0.636150
Epoch 117
Validation auc = 0.639437
Epoch 118
Validation auc = 0.639437
Epoch 119
Loss = 2.6300e-02, PNorm = 40.8559, GNorm = 0.6746, lr_0 = 8.2576e-04
Validation auc = 0.643192
Epoch 120
Validation auc = 0.642254
Epoch 121
Validation auc = 0.648357
Epoch 122
Validation auc = 0.649296
Epoch 123
Validation auc = 0.638028
Epoch 124
Loss = 6.2928e-02, PNorm = 40.9819, GNorm = 0.3006, lr_0 = 8.1870e-04
Validation auc = 0.648826
Epoch 125
Validation auc = 0.655399
Epoch 126
Validation auc = 0.662911
Epoch 127
Validation auc = 0.660094
Epoch 128
Validation auc = 0.650704
Epoch 129
Loss = 4.0647e-02, PNorm = 41.1043, GNorm = 0.7990, lr_0 = 8.1170e-04
Validation auc = 0.653052
Epoch 130
Validation auc = 0.663380
Epoch 131
Validation auc = 0.661033
Epoch 132
Validation auc = 0.657277
Epoch 133
Validation auc = 0.660094
Epoch 134
Loss = 1.5534e-02, PNorm = 41.2384, GNorm = 0.3927, lr_0 = 8.0476e-04
Validation auc = 0.661033
Epoch 135
Validation auc = 0.662441
Epoch 136
Validation auc = 0.663850
Epoch 137
Validation auc = 0.664319
Epoch 138
Validation auc = 0.661972
Epoch 139
Loss = 1.5548e-02, PNorm = 41.3362, GNorm = 0.2146, lr_0 = 7.9788e-04
Validation auc = 0.658685
Epoch 140
Validation auc = 0.659155
Epoch 141
Validation auc = 0.655399
Epoch 142
Validation auc = 0.646009
Epoch 143
Validation auc = 0.638028
Epoch 144
Loss = 4.3407e-02, PNorm = 41.4223, GNorm = 1.9991, lr_0 = 7.9106e-04
Validation auc = 0.644131
Epoch 145
Validation auc = 0.643662
Epoch 146
Validation auc = 0.645070
Epoch 147
Validation auc = 0.646948
Epoch 148
Validation auc = 0.649296
Epoch 149
Loss = 3.1285e-02, PNorm = 41.4972, GNorm = 1.5414, lr_0 = 7.8430e-04
Validation auc = 0.653521
Epoch 150
Validation auc = 0.655869
Epoch 151
Validation auc = 0.662911
Epoch 152
Validation auc = 0.666667
Epoch 153
Validation auc = 0.664319
Epoch 154
Loss = 1.6215e-02, PNorm = 41.5719, GNorm = 0.2265, lr_0 = 7.7759e-04
Validation auc = 0.666197
Epoch 155
Validation auc = 0.662911
Epoch 156
Validation auc = 0.653521
Epoch 157
Validation auc = 0.651643
Epoch 158
Validation auc = 0.656338
Epoch 159
Loss = 2.6283e-02, PNorm = 41.6533, GNorm = 0.2514, lr_0 = 7.7095e-04
Validation auc = 0.668075
Epoch 160
Validation auc = 0.667136
Epoch 161
Validation auc = 0.659624
Epoch 162
Validation auc = 0.661502
Epoch 163
Validation auc = 0.668075
Epoch 164
Loss = 1.3185e-02, PNorm = 41.7285, GNorm = 0.3969, lr_0 = 7.6436e-04
Validation auc = 0.668545
Epoch 165
Validation auc = 0.671831
Epoch 166
Validation auc = 0.667136
Epoch 167
Validation auc = 0.659155
Epoch 168
Validation auc = 0.656808
Epoch 169
Loss = 7.8504e-03, PNorm = 41.8027, GNorm = 0.2839, lr_0 = 7.5782e-04
Validation auc = 0.653991
Epoch 170
Validation auc = 0.653521
Epoch 171
Validation auc = 0.653991
Epoch 172
Validation auc = 0.658216
Epoch 173
Validation auc = 0.659155
Epoch 174
Loss = 3.3890e-02, PNorm = 41.8716, GNorm = 0.2266, lr_0 = 7.5134e-04
Validation auc = 0.667136
Epoch 175
Validation auc = 0.669484
Epoch 176
Validation auc = 0.660563
Epoch 177
Validation auc = 0.663380
Epoch 178
Validation auc = 0.661502
Epoch 179
Loss = 1.8214e-02, PNorm = 41.9508, GNorm = 0.5180, lr_0 = 7.4492e-04
Validation auc = 0.660563
Epoch 180
Validation auc = 0.661972
Epoch 181
Validation auc = 0.661502
Epoch 182
Validation auc = 0.661972
Epoch 183
Validation auc = 0.661502
Epoch 184
Loss = 1.8560e-02, PNorm = 42.0229, GNorm = 0.5629, lr_0 = 7.3855e-04
Validation auc = 0.664319
Epoch 185
Validation auc = 0.663850
Epoch 186
Validation auc = 0.666197
Epoch 187
Validation auc = 0.663850
Epoch 188
Validation auc = 0.664789
Epoch 189
Loss = 1.3693e-02, PNorm = 42.0755, GNorm = 1.4163, lr_0 = 7.3224e-04
Validation auc = 0.669014
Epoch 190
Validation auc = 0.676526
Epoch 191
Validation auc = 0.674178
Epoch 192
Validation auc = 0.669014
Epoch 193
Validation auc = 0.662911
Epoch 194
Loss = 6.1554e-03, PNorm = 42.1314, GNorm = 0.0387, lr_0 = 7.2598e-04
Validation auc = 0.662911
Epoch 195
Validation auc = 0.664319
Epoch 196
Validation auc = 0.666197
Epoch 197
Validation auc = 0.665258
Epoch 198
Validation auc = 0.666197
Epoch 199
Loss = 1.1375e-02, PNorm = 42.1963, GNorm = 0.3348, lr_0 = 7.1977e-04
Validation auc = 0.671362
Epoch 200
Validation auc = 0.675117
Epoch 201
Validation auc = 0.676995
Epoch 202
Validation auc = 0.676056
Epoch 203
Validation auc = 0.675117
Epoch 204
Loss = 3.4948e-03, PNorm = 42.2503, GNorm = 0.0555, lr_0 = 7.1362e-04
Validation auc = 0.675587
Epoch 205
Validation auc = 0.673709
Epoch 206
Validation auc = 0.672300
Epoch 207
Validation auc = 0.676526
Epoch 208
Validation auc = 0.671831
Epoch 209
Loss = 5.5000e-03, PNorm = 42.2940, GNorm = 0.0662, lr_0 = 7.0752e-04
Validation auc = 0.666197
Epoch 210
Validation auc = 0.664319
Epoch 211
Validation auc = 0.662441
Epoch 212
Validation auc = 0.663380
Epoch 213
Validation auc = 0.661033
Epoch 214
Loss = 4.0076e-03, PNorm = 42.3343, GNorm = 0.1820, lr_0 = 7.0147e-04
Validation auc = 0.660563
Epoch 215
Validation auc = 0.658685
Epoch 216
Validation auc = 0.659155
Epoch 217
Validation auc = 0.658685
Epoch 218
Validation auc = 0.658216
Epoch 219
Loss = 1.8465e-03, PNorm = 42.3685, GNorm = 0.1342, lr_0 = 6.9548e-04
Validation auc = 0.657277
Epoch 220
Validation auc = 0.656338
Epoch 221
Validation auc = 0.657277
Epoch 222
Validation auc = 0.658685
Epoch 223
Validation auc = 0.662441
Epoch 224
Loss = 4.9842e-03, PNorm = 42.3954, GNorm = 0.4694, lr_0 = 6.8953e-04
Validation auc = 0.663850
Epoch 225
Validation auc = 0.663380
Epoch 226
Validation auc = 0.661033
Epoch 227
Validation auc = 0.664789
Epoch 228
Validation auc = 0.664789
Epoch 229
Loss = 4.0019e-03, PNorm = 42.4240, GNorm = 0.1663, lr_0 = 6.8364e-04
Validation auc = 0.667136
Epoch 230
Validation auc = 0.669484
Epoch 231
Validation auc = 0.662441
Epoch 232
Validation auc = 0.655869
Epoch 233
Validation auc = 0.661033
Epoch 234
Loss = 2.7464e-03, PNorm = 42.4648, GNorm = 0.0437, lr_0 = 6.7779e-04
Validation auc = 0.670423
Epoch 235
Validation auc = 0.671831
Epoch 236
Validation auc = 0.667606
Epoch 237
Validation auc = 0.668075
Epoch 238
Validation auc = 0.674178
Epoch 239
Loss = 1.1676e-02, PNorm = 42.5185, GNorm = 0.8671, lr_0 = 6.7200e-04
Validation auc = 0.672770
Epoch 240
Validation auc = 0.661972
Epoch 241
Validation auc = 0.654930
Epoch 242
Validation auc = 0.658216
Epoch 243
Validation auc = 0.659155
Epoch 244
Loss = 2.0345e-02, PNorm = 42.6010, GNorm = 0.5341, lr_0 = 6.6625e-04
Validation auc = 0.649296
Epoch 245
Validation auc = 0.635681
Epoch 246
Validation auc = 0.636620
Epoch 247
Validation auc = 0.653991
Epoch 248
Validation auc = 0.667606
Epoch 249
Loss = 2.6253e-02, PNorm = 42.7241, GNorm = 1.0399, lr_0 = 6.6056e-04
Validation auc = 0.663380
Epoch 250
Validation auc = 0.658216
Epoch 251
Validation auc = 0.654930
Epoch 252
Validation auc = 0.655399
Epoch 253
Validation auc = 0.657746
Epoch 254
Loss = 5.0035e-02, PNorm = 42.8170, GNorm = 1.2347, lr_0 = 6.5491e-04
Validation auc = 0.661033
Epoch 255
Validation auc = 0.662441
Epoch 256
Validation auc = 0.656808
Epoch 257
Validation auc = 0.640376
Epoch 258
Validation auc = 0.637089
Epoch 259
Loss = 2.0801e-02, PNorm = 42.8860, GNorm = 0.4588, lr_0 = 6.4931e-04
Validation auc = 0.645540
Epoch 260
Validation auc = 0.661972
Epoch 261
Validation auc = 0.663380
Epoch 262
Validation auc = 0.662441
Epoch 263
Validation auc = 0.659624
Epoch 264
Loss = 2.5078e-02, PNorm = 42.9516, GNorm = 1.2883, lr_0 = 6.4376e-04
Validation auc = 0.666197
Epoch 265
Validation auc = 0.673709
Epoch 266
Validation auc = 0.675117
Epoch 267
Validation auc = 0.678404
Epoch 268
Validation auc = 0.676526
Epoch 269
Loss = 4.7789e-03, PNorm = 43.0013, GNorm = 0.1041, lr_0 = 6.3826e-04
Validation auc = 0.675117
Epoch 270
Validation auc = 0.669484
Epoch 271
Validation auc = 0.667606
Epoch 272
Validation auc = 0.666197
Epoch 273
Validation auc = 0.664319
Epoch 274
Loss = 2.1404e-03, PNorm = 43.0386, GNorm = 0.0389, lr_0 = 6.3280e-04
Validation auc = 0.667136
Epoch 275
Validation auc = 0.668075
Epoch 276
Validation auc = 0.669484
Epoch 277
Validation auc = 0.667606
Epoch 278
Validation auc = 0.667606
Epoch 279
Loss = 2.1014e-03, PNorm = 43.0646, GNorm = 0.0286, lr_0 = 6.2739e-04
Validation auc = 0.666197
Epoch 280
Validation auc = 0.666197
Epoch 281
Validation auc = 0.664789
Epoch 282
Validation auc = 0.663380
Epoch 283
Validation auc = 0.664319
Epoch 284
Loss = 1.3700e-03, PNorm = 43.0844, GNorm = 0.0582, lr_0 = 6.2203e-04
Validation auc = 0.665728
Epoch 285
Validation auc = 0.667136
Epoch 286
Validation auc = 0.669953
Epoch 287
Validation auc = 0.669484
Epoch 288
Validation auc = 0.670423
Epoch 289
Loss = 2.2897e-03, PNorm = 43.1030, GNorm = 0.0869, lr_0 = 6.1671e-04
Validation auc = 0.669953
Epoch 290
Validation auc = 0.669953
Epoch 291
Validation auc = 0.670423
Epoch 292
Validation auc = 0.671362
Epoch 293
Validation auc = 0.672300
Epoch 294
Loss = 1.9581e-03, PNorm = 43.1207, GNorm = 0.1610, lr_0 = 6.1144e-04
Validation auc = 0.673239
Epoch 295
Validation auc = 0.673709
Epoch 296
Validation auc = 0.674178
Epoch 297
Validation auc = 0.674648
Epoch 298
Validation auc = 0.673239
Epoch 299
Loss = 2.7809e-03, PNorm = 43.1374, GNorm = 0.0250, lr_0 = 6.0621e-04
Validation auc = 0.671831
Model 0 best validation auc = 0.756338 on epoch 5
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test auc = 0.747140
Ensemble test auc = 0.747140
Fold 1
Splitting data with seed 1
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.05, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=300, bias=False)
        (W_h): Linear(in_features=300, out_features=300, bias=False)
        (W_o): Linear(in_features=433, out_features=300, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.05, inplace=False)
    (1): Linear(in_features=2200, out_features=300, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.05, inplace=False)
    (4): Linear(in_features=300, out_features=1, bias=True)
  )
)
Number of parameters = 924,901
Moving model to cuda
Epoch 0
Validation auc = 0.665728
Epoch 1
Validation auc = 0.706103
Epoch 2
Validation auc = 0.760094
Epoch 3
Validation auc = 0.670423
Epoch 4
Loss = 8.1339e-01, PNorm = 37.1927, GNorm = 13.9106, lr_0 = 6.5000e-04
Validation auc = 0.672300
Epoch 5
Validation auc = 0.723005
Epoch 6
Validation auc = 0.659155
Epoch 7
Validation auc = 0.656808
Epoch 8
Validation auc = 0.722535
Epoch 9
Loss = 6.9411e-01, PNorm = 37.2530, GNorm = 14.7780, lr_0 = 9.9743e-04
Validation auc = 0.662441
Epoch 10
Validation auc = 0.644601
Epoch 11
Validation auc = 0.656808
Epoch 12
Validation auc = 0.655399
Epoch 13
Validation auc = 0.641315
Epoch 14
Loss = 5.0834e-01, PNorm = 37.3456, GNorm = 5.9209, lr_0 = 9.8890e-04
Validation auc = 0.642254
Epoch 15
Validation auc = 0.619249
Epoch 16
Validation auc = 0.637559
Epoch 17
Validation auc = 0.636150
Epoch 18
Validation auc = 0.603756
Epoch 19
Loss = 4.9201e-01, PNorm = 37.4582, GNorm = 1.3634, lr_0 = 9.8045e-04
Validation auc = 0.628638
Epoch 20
Validation auc = 0.630516
Epoch 21
Validation auc = 0.614554
Epoch 22
Validation auc = 0.630986
Epoch 23
Validation auc = 0.626761
Epoch 24
Loss = 5.0659e-01, PNorm = 37.5951, GNorm = 7.7047, lr_0 = 9.7207e-04
Validation auc = 0.630516
Epoch 25
Validation auc = 0.627230
Epoch 26
Validation auc = 0.629108
Epoch 27
Validation auc = 0.586854
Epoch 28
Validation auc = 0.552113
Epoch 29
Loss = 3.9642e-01, PNorm = 37.7453, GNorm = 2.1713, lr_0 = 9.6376e-04
Validation auc = 0.611268
Epoch 30
Validation auc = 0.616901
Epoch 31
Validation auc = 0.602817
Epoch 32
Validation auc = 0.615493
Epoch 33
Validation auc = 0.616432
Epoch 34
Loss = 2.9381e-01, PNorm = 37.9292, GNorm = 4.8563, lr_0 = 9.5552e-04
Validation auc = 0.610798
Epoch 35
Validation auc = 0.622535
Epoch 36
Validation auc = 0.600000
Epoch 37
Validation auc = 0.619249
Epoch 38
Validation auc = 0.623944
Epoch 39
Loss = 3.2985e-01, PNorm = 38.1097, GNorm = 2.0104, lr_0 = 9.4735e-04
Validation auc = 0.614085
Epoch 40
Validation auc = 0.631455
Epoch 41
Validation auc = 0.623474
Epoch 42
Validation auc = 0.625822
Epoch 43
Validation auc = 0.617371
Epoch 44
Loss = 2.0345e-01, PNorm = 38.3100, GNorm = 2.3846, lr_0 = 9.3925e-04
Validation auc = 0.613146
Epoch 45
Validation auc = 0.622535
Epoch 46
Validation auc = 0.615962
Epoch 47
Validation auc = 0.627230
Epoch 48
Validation auc = 0.617840
Epoch 49
Loss = 1.9714e-01, PNorm = 38.5153, GNorm = 3.1462, lr_0 = 9.3122e-04
Validation auc = 0.602817
Epoch 50
Validation auc = 0.610798
Epoch 51
Validation auc = 0.614554
Epoch 52
Validation auc = 0.626291
Epoch 53
Validation auc = 0.630986
Epoch 54
Loss = 2.8722e-01, PNorm = 38.7246, GNorm = 3.1030, lr_0 = 9.2326e-04
Validation auc = 0.629577
Epoch 55
Validation auc = 0.636150
Epoch 56
Validation auc = 0.619718
Epoch 57
Validation auc = 0.630986
Epoch 58
Validation auc = 0.633333
Epoch 59
Loss = 1.5712e-01, PNorm = 38.9265, GNorm = 2.4415, lr_0 = 9.1537e-04
Validation auc = 0.635211
Epoch 60
Validation auc = 0.634742
Epoch 61
Validation auc = 0.625352
Epoch 62
Validation auc = 0.616901
Epoch 63
Validation auc = 0.621596
Epoch 64
Loss = 1.5122e-01, PNorm = 39.1245, GNorm = 1.3498, lr_0 = 9.0754e-04
Validation auc = 0.625352
Epoch 65
Validation auc = 0.634272
Epoch 66
Validation auc = 0.629577
Epoch 67
Validation auc = 0.649296
Epoch 68
Validation auc = 0.652113
Epoch 69
Loss = 1.3508e-01, PNorm = 39.3279, GNorm = 4.0371, lr_0 = 8.9978e-04
Validation auc = 0.645540
Epoch 70
Validation auc = 0.644601
Epoch 71
Validation auc = 0.610329
Epoch 72
Validation auc = 0.623005
Epoch 73
Validation auc = 0.624883
Epoch 74
Loss = 1.1873e-01, PNorm = 39.5081, GNorm = 0.9181, lr_0 = 8.9209e-04
Validation auc = 0.617371
Epoch 75
Validation auc = 0.614085
Epoch 76
Validation auc = 0.610798
Epoch 77
Validation auc = 0.602817
Epoch 78
Validation auc = 0.607512
Epoch 79
Loss = 1.2542e-01, PNorm = 39.6943, GNorm = 1.4272, lr_0 = 8.8447e-04
Validation auc = 0.622066
Epoch 80
Validation auc = 0.632864
Epoch 81
Validation auc = 0.643662
Epoch 82
Validation auc = 0.649296
Epoch 83
Validation auc = 0.628169
Epoch 84
Loss = 1.2365e-01, PNorm = 39.8632, GNorm = 3.3216, lr_0 = 8.7691e-04
Validation auc = 0.623944
Epoch 85
Validation auc = 0.622535
Epoch 86
Validation auc = 0.627230
Epoch 87
Validation auc = 0.629577
Epoch 88
Validation auc = 0.631925
Epoch 89
Loss = 1.0283e-01, PNorm = 40.0519, GNorm = 1.3935, lr_0 = 8.6941e-04
Validation auc = 0.632394
Epoch 90
Validation auc = 0.634272
Epoch 91
Validation auc = 0.615493
Epoch 92
Validation auc = 0.601408
Epoch 93
Validation auc = 0.603286
Epoch 94
Loss = 7.6926e-02, PNorm = 40.2340, GNorm = 0.9548, lr_0 = 8.6198e-04
Validation auc = 0.614085
Epoch 95
Validation auc = 0.621127
Epoch 96
Validation auc = 0.630986
Epoch 97
Validation auc = 0.636150
Epoch 98
Validation auc = 0.644601
Epoch 99
Loss = 4.3998e-02, PNorm = 40.4120, GNorm = 1.2279, lr_0 = 8.5461e-04
Validation auc = 0.646479
Epoch 100
Validation auc = 0.646009
Epoch 101
Validation auc = 0.645540
Epoch 102
Validation auc = 0.646009
Epoch 103
Validation auc = 0.645540
Epoch 104
Loss = 6.4538e-02, PNorm = 40.5429, GNorm = 0.7570, lr_0 = 8.4730e-04
Validation auc = 0.638498
Epoch 105
Validation auc = 0.632864
Epoch 106
Validation auc = 0.631455
Epoch 107
Validation auc = 0.642723
Epoch 108
Validation auc = 0.650704
Epoch 109
Loss = 4.1486e-02, PNorm = 40.6828, GNorm = 1.3232, lr_0 = 8.4006e-04
Validation auc = 0.655399
Epoch 110
Validation auc = 0.667606
Epoch 111
Validation auc = 0.652582
Epoch 112
Validation auc = 0.633333
Epoch 113
Validation auc = 0.617840
Epoch 114
Loss = 5.8608e-02, PNorm = 40.8082, GNorm = 0.9238, lr_0 = 8.3288e-04
Validation auc = 0.612207
Epoch 115
Validation auc = 0.610329
Epoch 116
Validation auc = 0.612676
Epoch 117
Validation auc = 0.618779
Epoch 118
Validation auc = 0.614085
Epoch 119
Loss = 6.2444e-02, PNorm = 40.9546, GNorm = 1.0766, lr_0 = 8.2576e-04
Validation auc = 0.613615
Epoch 120
Validation auc = 0.609390
Epoch 121
Validation auc = 0.612207
Epoch 122
Validation auc = 0.613146
Epoch 123
Validation auc = 0.616901
Epoch 124
Loss = 2.5057e-02, PNorm = 41.0881, GNorm = 0.1930, lr_0 = 8.1870e-04
Validation auc = 0.622066
Epoch 125
Validation auc = 0.623944
Epoch 126
Validation auc = 0.628169
Epoch 127
Validation auc = 0.630516
Epoch 128
Validation auc = 0.635211
Epoch 129
Loss = 2.3845e-02, PNorm = 41.2030, GNorm = 0.6208, lr_0 = 8.1170e-04
Validation auc = 0.644601
Epoch 130
Validation auc = 0.646948
Epoch 131
Validation auc = 0.648357
Epoch 132
Validation auc = 0.652113
Epoch 133
Validation auc = 0.660563
Epoch 134
Loss = 5.5829e-02, PNorm = 41.3071, GNorm = 1.0378, lr_0 = 8.0476e-04
Validation auc = 0.660563
Epoch 135
Validation auc = 0.654930
Epoch 136
Validation auc = 0.649765
Epoch 137
Validation auc = 0.653052
Epoch 138
Validation auc = 0.648357
Epoch 139
Loss = 1.3279e-02, PNorm = 41.3995, GNorm = 0.2098, lr_0 = 7.9788e-04
Validation auc = 0.644601
Epoch 140
Validation auc = 0.643662
Epoch 141
Validation auc = 0.641315
Epoch 142
Validation auc = 0.642254
Epoch 143
Validation auc = 0.649296
Epoch 144
Loss = 1.5023e-02, PNorm = 41.4846, GNorm = 0.3693, lr_0 = 7.9106e-04
Validation auc = 0.652582
Epoch 145
Validation auc = 0.656338
Epoch 146
Validation auc = 0.652582
Epoch 147
Validation auc = 0.647887
Epoch 148
Validation auc = 0.647887
Epoch 149
Loss = 2.0865e-02, PNorm = 41.5594, GNorm = 0.8865, lr_0 = 7.8430e-04
Validation auc = 0.652113
Epoch 150
Validation auc = 0.655399
Epoch 151
Validation auc = 0.652582
Epoch 152
Validation auc = 0.652113
Epoch 153
Validation auc = 0.650704
Epoch 154
Loss = 2.7542e-02, PNorm = 41.6329, GNorm = 0.2620, lr_0 = 7.7759e-04
Validation auc = 0.654460
Epoch 155
Validation auc = 0.655869
Epoch 156
Validation auc = 0.661972
Epoch 157
Validation auc = 0.659155
Epoch 158
Validation auc = 0.660094
Epoch 159
Loss = 2.4733e-02, PNorm = 41.7086, GNorm = 0.8798, lr_0 = 7.7095e-04
Validation auc = 0.655869
Epoch 160
Validation auc = 0.651174
Epoch 161
Validation auc = 0.649765
Epoch 162
Validation auc = 0.646009
Epoch 163
Validation auc = 0.644601
Epoch 164
Loss = 2.7336e-02, PNorm = 41.7890, GNorm = 0.8623, lr_0 = 7.6436e-04
Validation auc = 0.636620
Epoch 165
Validation auc = 0.643662
Epoch 166
Validation auc = 0.642723
Epoch 167
Validation auc = 0.639906
Epoch 168
Validation auc = 0.633803
Epoch 169
Loss = 5.2413e-03, PNorm = 41.8823, GNorm = 0.1716, lr_0 = 7.5782e-04
Validation auc = 0.631925
Epoch 170
Validation auc = 0.632864
Epoch 171
Validation auc = 0.650235
Epoch 172
Validation auc = 0.653052
Epoch 173
Validation auc = 0.651643
Epoch 174
Loss = 2.8570e-02, PNorm = 41.9671, GNorm = 0.6649, lr_0 = 7.5134e-04
Validation auc = 0.650704
Epoch 175
Validation auc = 0.650704
Epoch 176
Validation auc = 0.647887
Epoch 177
Validation auc = 0.646948
Epoch 178
Validation auc = 0.645070
Epoch 179
Loss = 2.5765e-02, PNorm = 42.0501, GNorm = 0.5191, lr_0 = 7.4492e-04
Validation auc = 0.641315
Epoch 180
Validation auc = 0.643192
Epoch 181
Validation auc = 0.647887
Epoch 182
Validation auc = 0.651174
Epoch 183
Validation auc = 0.653521
Epoch 184
Loss = 1.0426e-02, PNorm = 42.1349, GNorm = 0.4293, lr_0 = 7.3855e-04
Validation auc = 0.656808
Epoch 185
Validation auc = 0.654460
Epoch 186
Validation auc = 0.659155
Epoch 187
Validation auc = 0.659624
Epoch 188
Validation auc = 0.666197
Epoch 189
Loss = 1.5398e-02, PNorm = 42.2084, GNorm = 0.4083, lr_0 = 7.3224e-04
Validation auc = 0.662911
Epoch 190
Validation auc = 0.663850
Epoch 191
Validation auc = 0.656338
Epoch 192
Validation auc = 0.654460
Epoch 193
Validation auc = 0.660563
Epoch 194
Loss = 1.4233e-02, PNorm = 42.2753, GNorm = 0.6013, lr_0 = 7.2598e-04
Validation auc = 0.664789
Epoch 195
Validation auc = 0.656808
Epoch 196
Validation auc = 0.654460
Epoch 197
Validation auc = 0.653521
Epoch 198
Validation auc = 0.653052
Epoch 199
Loss = 4.8795e-03, PNorm = 42.3374, GNorm = 0.1199, lr_0 = 7.1977e-04
Validation auc = 0.651643
Epoch 200
Validation auc = 0.649296
Epoch 201
Validation auc = 0.646948
Epoch 202
Validation auc = 0.646009
Epoch 203
Validation auc = 0.641315
Epoch 204
Loss = 7.1903e-03, PNorm = 42.3957, GNorm = 0.3023, lr_0 = 7.1362e-04
Validation auc = 0.642723
Epoch 205
Validation auc = 0.643662
Epoch 206
Validation auc = 0.644131
Epoch 207
Validation auc = 0.651174
Epoch 208
Validation auc = 0.654930
Epoch 209
Loss = 6.2776e-03, PNorm = 42.4435, GNorm = 0.2696, lr_0 = 7.0752e-04
Validation auc = 0.654930
Epoch 210
Validation auc = 0.652582
Epoch 211
Validation auc = 0.650235
Epoch 212
Validation auc = 0.649765
Epoch 213
Validation auc = 0.651643
Epoch 214
Loss = 3.2412e-03, PNorm = 42.4845, GNorm = 0.1241, lr_0 = 7.0147e-04
Validation auc = 0.655399
Epoch 215
Validation auc = 0.655869
Epoch 216
Validation auc = 0.652582
Epoch 217
Validation auc = 0.653052
Epoch 218
Validation auc = 0.652113
Epoch 219
Loss = 6.7184e-03, PNorm = 42.5238, GNorm = 0.2896, lr_0 = 6.9548e-04
Validation auc = 0.649765
Epoch 220
Validation auc = 0.651174
Epoch 221
Validation auc = 0.656808
Epoch 222
Validation auc = 0.655869
Epoch 223
Validation auc = 0.654460
Epoch 224
Loss = 9.5285e-03, PNorm = 42.5627, GNorm = 0.0620, lr_0 = 6.8953e-04
Validation auc = 0.652582
Epoch 225
Validation auc = 0.649296
Epoch 226
Validation auc = 0.646009
Epoch 227
Validation auc = 0.646948
Epoch 228
Validation auc = 0.649296
Epoch 229
Loss = 4.4288e-03, PNorm = 42.5981, GNorm = 0.2140, lr_0 = 6.8364e-04
Validation auc = 0.650704
Epoch 230
Validation auc = 0.652582
Epoch 231
Validation auc = 0.655399
Epoch 232
Validation auc = 0.654930
Epoch 233
Validation auc = 0.655399
Epoch 234
Loss = 2.0129e-03, PNorm = 42.6295, GNorm = 0.0338, lr_0 = 6.7779e-04
Validation auc = 0.655399
Epoch 235
Validation auc = 0.656338
Epoch 236
Validation auc = 0.659155
Epoch 237
Validation auc = 0.659155
Epoch 238
Validation auc = 0.653521
Epoch 239
Loss = 3.1136e-03, PNorm = 42.6619, GNorm = 0.0583, lr_0 = 6.7200e-04
Validation auc = 0.653052
Epoch 240
Validation auc = 0.644131
Epoch 241
Validation auc = 0.643662
Epoch 242
Validation auc = 0.646479
Epoch 243
Validation auc = 0.649296
Epoch 244
Loss = 6.6171e-03, PNorm = 42.6977, GNorm = 0.3505, lr_0 = 6.6625e-04
Validation auc = 0.651643
Epoch 245
Validation auc = 0.649296
Epoch 246
Validation auc = 0.646479
Epoch 247
Validation auc = 0.641315
Epoch 248
Validation auc = 0.640376
Epoch 249
Loss = 1.4503e-03, PNorm = 42.7349, GNorm = 0.0362, lr_0 = 6.6056e-04
Validation auc = 0.642254
Epoch 250
Validation auc = 0.644131
Epoch 251
Validation auc = 0.648826
Epoch 252
Validation auc = 0.652582
Epoch 253
Validation auc = 0.653052
Epoch 254
Loss = 4.6000e-03, PNorm = 42.7644, GNorm = 0.1265, lr_0 = 6.5491e-04
Validation auc = 0.653052
Epoch 255
Validation auc = 0.654930
Epoch 256
Validation auc = 0.656338
Epoch 257
Validation auc = 0.656808
Epoch 258
Validation auc = 0.660563
Epoch 259
Loss = 2.8146e-03, PNorm = 42.7931, GNorm = 0.1356, lr_0 = 6.4931e-04
Validation auc = 0.661972
Epoch 260
Validation auc = 0.661972
Epoch 261
Validation auc = 0.662441
Epoch 262
Validation auc = 0.662441
Epoch 263
Validation auc = 0.660094
Epoch 264
Loss = 2.5488e-03, PNorm = 42.8165, GNorm = 0.0253, lr_0 = 6.4376e-04
Validation auc = 0.661033
Epoch 265
Validation auc = 0.660094
Epoch 266
Validation auc = 0.660563
Epoch 267
Validation auc = 0.661972
Epoch 268
Validation auc = 0.661972
Epoch 269
Loss = 1.6207e-03, PNorm = 42.8377, GNorm = 0.0981, lr_0 = 6.3826e-04
Validation auc = 0.661972
Epoch 270
Validation auc = 0.661033
Epoch 271
Validation auc = 0.661033
Epoch 272
Validation auc = 0.660094
Epoch 273
Validation auc = 0.661033
Epoch 274
Loss = 1.8699e-03, PNorm = 42.8568, GNorm = 0.1100, lr_0 = 6.3280e-04
Validation auc = 0.661033
Epoch 275
Validation auc = 0.660563
Epoch 276
Validation auc = 0.661972
Epoch 277
Validation auc = 0.663380
Epoch 278
Validation auc = 0.661033
Epoch 279
Loss = 8.6561e-04, PNorm = 42.8802, GNorm = 0.0162, lr_0 = 6.2739e-04
Validation auc = 0.659155
Epoch 280
Validation auc = 0.660094
Epoch 281
Validation auc = 0.661033
Epoch 282
Validation auc = 0.658216
Epoch 283
Validation auc = 0.661033
Epoch 284
Loss = 8.7217e-04, PNorm = 42.9006, GNorm = 0.0314, lr_0 = 6.2203e-04
Validation auc = 0.661972
Epoch 285
Validation auc = 0.661033
Epoch 286
Validation auc = 0.663850
Epoch 287
Validation auc = 0.660563
Epoch 288
Validation auc = 0.659624
Epoch 289
Loss = 1.7162e-03, PNorm = 42.9247, GNorm = 0.0571, lr_0 = 6.1671e-04
Validation auc = 0.656338
Epoch 290
Validation auc = 0.651174
Epoch 291
Validation auc = 0.649296
Epoch 292
Validation auc = 0.648357
Epoch 293
Validation auc = 0.650235
Epoch 294
Loss = 7.7801e-04, PNorm = 42.9467, GNorm = 0.0251, lr_0 = 6.1144e-04
Validation auc = 0.651643
Epoch 295
Validation auc = 0.651643
Epoch 296
Validation auc = 0.651643
Epoch 297
Validation auc = 0.650235
Epoch 298
Validation auc = 0.650704
Epoch 299
Loss = 9.6662e-04, PNorm = 42.9678, GNorm = 0.0190, lr_0 = 6.0621e-04
Validation auc = 0.649296
Model 0 best validation auc = 0.760094 on epoch 2
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test auc = 0.720650
Ensemble test auc = 0.720650
Fold 2
Splitting data with seed 2
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.05, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=300, bias=False)
        (W_h): Linear(in_features=300, out_features=300, bias=False)
        (W_o): Linear(in_features=433, out_features=300, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.05, inplace=False)
    (1): Linear(in_features=2200, out_features=300, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.05, inplace=False)
    (4): Linear(in_features=300, out_features=1, bias=True)
  )
)
Number of parameters = 924,901
Moving model to cuda
Epoch 0
Validation auc = 0.660094
Epoch 1
Validation auc = 0.673709
Epoch 2
Validation auc = 0.706573
Epoch 3
Validation auc = 0.676526
Epoch 4
Loss = 6.8953e-01, PNorm = 37.1919, GNorm = 6.6953, lr_0 = 6.5000e-04
Validation auc = 0.721596
Epoch 5
Validation auc = 0.703756
Epoch 6
Validation auc = 0.683099
Epoch 7
Validation auc = 0.684507
Epoch 8
Validation auc = 0.646479
Epoch 9
Loss = 6.1380e-01, PNorm = 37.2565, GNorm = 2.8158, lr_0 = 9.9743e-04
Validation auc = 0.634272
Epoch 10
Validation auc = 0.637559
Epoch 11
Validation auc = 0.610798
Epoch 12
Validation auc = 0.634272
Epoch 13
Validation auc = 0.643662
Epoch 14
Loss = 4.3813e-01, PNorm = 37.3645, GNorm = 5.4247, lr_0 = 9.8890e-04
Validation auc = 0.652113
Epoch 15
Validation auc = 0.664319
Epoch 16
Validation auc = 0.640845
Epoch 17
Validation auc = 0.642254
Epoch 18
Validation auc = 0.631925
Epoch 19
Loss = 5.7712e-01, PNorm = 37.4775, GNorm = 12.4577, lr_0 = 9.8045e-04
Validation auc = 0.623005
Epoch 20
Validation auc = 0.637089
Epoch 21
Validation auc = 0.636150
Epoch 22
Validation auc = 0.575587
Epoch 23
Validation auc = 0.605634
Epoch 24
Loss = 4.2401e-01, PNorm = 37.5977, GNorm = 4.9295, lr_0 = 9.7207e-04
Validation auc = 0.633333
Epoch 25
Validation auc = 0.629108
Epoch 26
Validation auc = 0.614085
Epoch 27
Validation auc = 0.615493
Epoch 28
Validation auc = 0.622066
Epoch 29
Loss = 4.3532e-01, PNorm = 37.7340, GNorm = 1.2776, lr_0 = 9.6376e-04
Validation auc = 0.609390
Epoch 30
Validation auc = 0.621596
Epoch 31
Validation auc = 0.627700
Epoch 32
Validation auc = 0.611737
Epoch 33
Validation auc = 0.623474
Epoch 34
Loss = 3.5128e-01, PNorm = 37.8850, GNorm = 2.3604, lr_0 = 9.5552e-04
Validation auc = 0.629577
Epoch 35
Validation auc = 0.630516
Epoch 36
Validation auc = 0.631455
Epoch 37
Validation auc = 0.630516
Epoch 38
Validation auc = 0.637089
Epoch 39
Loss = 3.2408e-01, PNorm = 38.0605, GNorm = 2.4922, lr_0 = 9.4735e-04
Validation auc = 0.633803
Epoch 40
Validation auc = 0.628169
Epoch 41
Validation auc = 0.619718
Epoch 42
Validation auc = 0.637089
Epoch 43
Validation auc = 0.625352
Epoch 44
Loss = 2.5212e-01, PNorm = 38.2487, GNorm = 3.4969, lr_0 = 9.3925e-04
Validation auc = 0.630986
Epoch 45
Validation auc = 0.633333
Epoch 46
Validation auc = 0.629577
Epoch 47
Validation auc = 0.626291
Epoch 48
Validation auc = 0.620657
Epoch 49
Loss = 2.7324e-01, PNorm = 38.4573, GNorm = 2.1572, lr_0 = 9.3122e-04
Validation auc = 0.636150
Epoch 50
Validation auc = 0.635681
Epoch 51
Validation auc = 0.630986
Epoch 52
Validation auc = 0.643662
Epoch 53
Validation auc = 0.627700
Epoch 54
Loss = 2.5633e-01, PNorm = 38.6677, GNorm = 5.8590, lr_0 = 9.2326e-04
Validation auc = 0.636150
Epoch 55
Validation auc = 0.637559
Epoch 56
Validation auc = 0.630047
Epoch 57
Validation auc = 0.632394
Epoch 58
Validation auc = 0.632864
Epoch 59
Loss = 1.9046e-01, PNorm = 38.8773, GNorm = 1.5063, lr_0 = 9.1537e-04
Validation auc = 0.622535
Epoch 60
Validation auc = 0.620188
Epoch 61
Validation auc = 0.629577
Epoch 62
Validation auc = 0.624413
Epoch 63
Validation auc = 0.615493
Epoch 64
Loss = 1.4045e-01, PNorm = 39.0944, GNorm = 2.6445, lr_0 = 9.0754e-04
Validation auc = 0.619718
Epoch 65
Validation auc = 0.619249
Epoch 66
Validation auc = 0.627230
Epoch 67
Validation auc = 0.632864
Epoch 68
Validation auc = 0.631925
Epoch 69
Loss = 1.0616e-01, PNorm = 39.2747, GNorm = 3.0903, lr_0 = 8.9978e-04
Validation auc = 0.632394
Epoch 70
Validation auc = 0.631455
Epoch 71
Validation auc = 0.621596
Epoch 72
Validation auc = 0.612676
Epoch 73
Validation auc = 0.619249
Epoch 74
Loss = 9.4049e-02, PNorm = 39.4609, GNorm = 2.6392, lr_0 = 8.9209e-04
Validation auc = 0.623474
Epoch 75
Validation auc = 0.634742
Epoch 76
Validation auc = 0.632394
Epoch 77
Validation auc = 0.607512
Epoch 78
Validation auc = 0.625352
Epoch 79
Loss = 1.5759e-01, PNorm = 39.6332, GNorm = 3.4349, lr_0 = 8.8447e-04
Validation auc = 0.625352
Epoch 80
Validation auc = 0.617840
Epoch 81
Validation auc = 0.623474
Epoch 82
Validation auc = 0.633333
Epoch 83
Validation auc = 0.629108
Epoch 84
Loss = 1.3929e-01, PNorm = 39.8491, GNorm = 3.4335, lr_0 = 8.7691e-04
Validation auc = 0.610329
Epoch 85
Validation auc = 0.625352
Epoch 86
Validation auc = 0.629108
Epoch 87
Validation auc = 0.629108
Epoch 88
Validation auc = 0.624883
Epoch 89
Loss = 1.4342e-01, PNorm = 40.0417, GNorm = 4.8633, lr_0 = 8.6941e-04
Validation auc = 0.636150
Epoch 90
Validation auc = 0.647418
Epoch 91
Validation auc = 0.647418
Epoch 92
Validation auc = 0.639437
Epoch 93
Validation auc = 0.647418
Epoch 94
Loss = 6.4475e-02, PNorm = 40.2175, GNorm = 1.2484, lr_0 = 8.6198e-04
Validation auc = 0.652582
Epoch 95
Validation auc = 0.650704
Epoch 96
Validation auc = 0.644131
Epoch 97
Validation auc = 0.631455
Epoch 98
Validation auc = 0.627700
Epoch 99
Loss = 4.7983e-02, PNorm = 40.3675, GNorm = 0.5371, lr_0 = 8.5461e-04
Validation auc = 0.630047
Epoch 100
Validation auc = 0.638498
Epoch 101
Validation auc = 0.641784
Epoch 102
Validation auc = 0.638498
Epoch 103
Validation auc = 0.644131
Epoch 104
Loss = 7.7523e-02, PNorm = 40.4891, GNorm = 0.7661, lr_0 = 8.4730e-04
Validation auc = 0.647418
Epoch 105
Validation auc = 0.648826
Epoch 106
Validation auc = 0.649765
Epoch 107
Validation auc = 0.647418
Epoch 108
Validation auc = 0.652582
Epoch 109
Loss = 4.3864e-02, PNorm = 40.6127, GNorm = 0.9046, lr_0 = 8.4006e-04
Validation auc = 0.654460
Epoch 110
Validation auc = 0.651174
Epoch 111
Validation auc = 0.646479
Epoch 112
Validation auc = 0.645540
Epoch 113
Validation auc = 0.646479
Epoch 114
Loss = 3.1304e-02, PNorm = 40.7267, GNorm = 0.6212, lr_0 = 8.3288e-04
Validation auc = 0.648826
Epoch 115
Validation auc = 0.656338
Epoch 116
Validation auc = 0.661502
Epoch 117
Validation auc = 0.665728
Epoch 118
Validation auc = 0.661502
Epoch 119
Loss = 1.3387e-02, PNorm = 40.8365, GNorm = 0.2170, lr_0 = 8.2576e-04
Validation auc = 0.658685
Epoch 120
Validation auc = 0.657746
Epoch 121
Validation auc = 0.658685
Epoch 122
Validation auc = 0.661033
Epoch 123
Validation auc = 0.662441
Epoch 124
Loss = 4.9043e-02, PNorm = 40.9276, GNorm = 0.7820, lr_0 = 8.1870e-04
Validation auc = 0.656808
Epoch 125
Validation auc = 0.655869
Epoch 126
Validation auc = 0.653521
Epoch 127
Validation auc = 0.652582
Epoch 128
Validation auc = 0.652113
Epoch 129
Loss = 3.4683e-02, PNorm = 41.0259, GNorm = 0.3491, lr_0 = 8.1170e-04
Validation auc = 0.659155
Epoch 130
Validation auc = 0.655869
Epoch 131
Validation auc = 0.656808
Epoch 132
Validation auc = 0.646948
Epoch 133
Validation auc = 0.646948
Epoch 134
Loss = 3.2648e-02, PNorm = 41.1130, GNorm = 0.6234, lr_0 = 8.0476e-04
Validation auc = 0.651174
Epoch 135
Validation auc = 0.659624
Epoch 136
Validation auc = 0.665728
Epoch 137
Validation auc = 0.659624
Epoch 138
Validation auc = 0.656338
Epoch 139
Loss = 5.3807e-02, PNorm = 41.1989, GNorm = 0.9734, lr_0 = 7.9788e-04
Validation auc = 0.663850
Epoch 140
Validation auc = 0.664319
Epoch 141
Validation auc = 0.650704
Epoch 142
Validation auc = 0.645070
Epoch 143
Validation auc = 0.644131
Epoch 144
Loss = 4.2183e-02, PNorm = 41.3050, GNorm = 1.0176, lr_0 = 7.9106e-04
Validation auc = 0.641784
Epoch 145
Validation auc = 0.641784
Epoch 146
Validation auc = 0.642254
Epoch 147
Validation auc = 0.646948
Epoch 148
Validation auc = 0.656808
Epoch 149
Loss = 2.4456e-02, PNorm = 41.4065, GNorm = 0.6534, lr_0 = 7.8430e-04
Validation auc = 0.656808
Epoch 150
Validation auc = 0.656808
Epoch 151
Validation auc = 0.658685
Epoch 152
Validation auc = 0.660563
Epoch 153
Validation auc = 0.659624
Epoch 154
Loss = 2.5040e-02, PNorm = 41.4881, GNorm = 0.2752, lr_0 = 7.7759e-04
Validation auc = 0.656338
Epoch 155
Validation auc = 0.657746
Epoch 156
Validation auc = 0.654930
Epoch 157
Validation auc = 0.652113
Epoch 158
Validation auc = 0.650235
Epoch 159
Loss = 1.1870e-02, PNorm = 41.5612, GNorm = 0.0852, lr_0 = 7.7095e-04
Validation auc = 0.651174
Epoch 160
Validation auc = 0.651174
Epoch 161
Validation auc = 0.648357
Epoch 162
Validation auc = 0.647418
Epoch 163
Validation auc = 0.651643
Epoch 164
Loss = 1.9258e-02, PNorm = 41.6285, GNorm = 0.1159, lr_0 = 7.6436e-04
Validation auc = 0.657277
Epoch 165
Validation auc = 0.659624
Epoch 166
Validation auc = 0.661033
Epoch 167
Validation auc = 0.661972
Epoch 168
Validation auc = 0.660563
Epoch 169
Loss = 6.6075e-03, PNorm = 41.6931, GNorm = 0.2751, lr_0 = 7.5782e-04
Validation auc = 0.658216
Epoch 170
Validation auc = 0.665728
Epoch 171
Validation auc = 0.668545
Epoch 172
Validation auc = 0.669953
Epoch 173
Validation auc = 0.667606
Epoch 174
Loss = 1.0075e-02, PNorm = 41.7473, GNorm = 0.2570, lr_0 = 7.5134e-04
Validation auc = 0.664319
Epoch 175
Validation auc = 0.656808
Epoch 176
Validation auc = 0.652113
Epoch 177
Validation auc = 0.649765
Epoch 178
Validation auc = 0.653052
Epoch 179
Loss = 6.1089e-03, PNorm = 41.8021, GNorm = 0.1465, lr_0 = 7.4492e-04
Validation auc = 0.654930
Epoch 180
Validation auc = 0.654460
Epoch 181
Validation auc = 0.652113
Epoch 182
Validation auc = 0.652113
Epoch 183
Validation auc = 0.655399
Epoch 184
Loss = 7.7713e-03, PNorm = 41.8567, GNorm = 0.0853, lr_0 = 7.3855e-04
Validation auc = 0.664789
Epoch 185
Validation auc = 0.669953
Epoch 186
Validation auc = 0.668075
Epoch 187
Validation auc = 0.662911
Epoch 188
Validation auc = 0.663850
Epoch 189
Loss = 2.0295e-02, PNorm = 41.9095, GNorm = 1.1927, lr_0 = 7.3224e-04
Validation auc = 0.670892
Epoch 190
Validation auc = 0.668075
Epoch 191
Validation auc = 0.663380
Epoch 192
Validation auc = 0.652582
Epoch 193
Validation auc = 0.654930
Epoch 194
Loss = 6.0847e-03, PNorm = 41.9650, GNorm = 0.4762, lr_0 = 7.2598e-04
Validation auc = 0.658216
Epoch 195
Validation auc = 0.659155
Epoch 196
Validation auc = 0.662441
Epoch 197
Validation auc = 0.665258
Epoch 198
Validation auc = 0.665258
Epoch 199
Loss = 1.1881e-02, PNorm = 42.0091, GNorm = 0.5564, lr_0 = 7.1977e-04
Validation auc = 0.665728
Epoch 200
Validation auc = 0.660563
Epoch 201
Validation auc = 0.661972
Epoch 202
Validation auc = 0.657746
Epoch 203
Validation auc = 0.661502
Epoch 204
Loss = 5.9399e-03, PNorm = 42.0498, GNorm = 0.1607, lr_0 = 7.1362e-04
Validation auc = 0.661033
Epoch 205
Validation auc = 0.661033
Epoch 206
Validation auc = 0.661033
Epoch 207
Validation auc = 0.676526
Epoch 208
Validation auc = 0.681690
Epoch 209
Loss = 6.0381e-03, PNorm = 42.0889, GNorm = 0.3424, lr_0 = 7.0752e-04
Validation auc = 0.676526
Epoch 210
Validation auc = 0.669953
Epoch 211
Validation auc = 0.669953
Epoch 212
Validation auc = 0.670892
Epoch 213
Validation auc = 0.670423
Epoch 214
Loss = 1.3209e-02, PNorm = 42.1266, GNorm = 0.8382, lr_0 = 7.0147e-04
Validation auc = 0.671831
Epoch 215
Validation auc = 0.666667
Epoch 216
Validation auc = 0.662911
Epoch 217
Validation auc = 0.656338
Epoch 218
Validation auc = 0.654460
Epoch 219
Loss = 3.2127e-03, PNorm = 42.1636, GNorm = 0.0869, lr_0 = 6.9548e-04
Validation auc = 0.655399
Epoch 220
Validation auc = 0.661033
Epoch 221
Validation auc = 0.668075
Epoch 222
Validation auc = 0.668075
Epoch 223
Validation auc = 0.666197
Epoch 224
Loss = 5.2385e-03, PNorm = 42.2102, GNorm = 0.2251, lr_0 = 6.8953e-04
Validation auc = 0.664789
Epoch 225
Validation auc = 0.664319
Epoch 226
Validation auc = 0.668075
Epoch 227
Validation auc = 0.675117
Epoch 228
Validation auc = 0.678404
Epoch 229
Loss = 5.7372e-03, PNorm = 42.2549, GNorm = 0.3235, lr_0 = 6.8364e-04
Validation auc = 0.677465
Epoch 230
Validation auc = 0.675587
Epoch 231
Validation auc = 0.669953
Epoch 232
Validation auc = 0.668545
Epoch 233
Validation auc = 0.670423
Epoch 234
Loss = 1.6703e-03, PNorm = 42.3007, GNorm = 0.0413, lr_0 = 6.7779e-04
Validation auc = 0.669953
Epoch 235
Validation auc = 0.668545
Epoch 236
Validation auc = 0.669484
Epoch 237
Validation auc = 0.669014
Epoch 238
Validation auc = 0.669014
Epoch 239
Loss = 2.4197e-03, PNorm = 42.3323, GNorm = 0.1325, lr_0 = 6.7200e-04
Validation auc = 0.669484
Epoch 240
Validation auc = 0.670892
Epoch 241
Validation auc = 0.675587
Epoch 242
Validation auc = 0.676995
Epoch 243
Validation auc = 0.676526
Epoch 244
Loss = 3.1082e-03, PNorm = 42.3557, GNorm = 0.0698, lr_0 = 6.6625e-04
Validation auc = 0.676056
Epoch 245
Validation auc = 0.674178
Epoch 246
Validation auc = 0.667136
Epoch 247
Validation auc = 0.660563
Epoch 248
Validation auc = 0.661502
Epoch 249
Loss = 3.2284e-03, PNorm = 42.3868, GNorm = 0.0468, lr_0 = 6.6056e-04
Validation auc = 0.665728
Epoch 250
Validation auc = 0.666197
Epoch 251
Validation auc = 0.668545
Epoch 252
Validation auc = 0.669014
Epoch 253
Validation auc = 0.667136
Epoch 254
Loss = 2.6566e-03, PNorm = 42.4162, GNorm = 0.1341, lr_0 = 6.5491e-04
Validation auc = 0.665728
Epoch 255
Validation auc = 0.666197
Epoch 256
Validation auc = 0.669014
Epoch 257
Validation auc = 0.668545
Epoch 258
Validation auc = 0.671831
Epoch 259
Loss = 1.1717e-03, PNorm = 42.4401, GNorm = 0.0361, lr_0 = 6.4931e-04
Validation auc = 0.674648
Epoch 260
Validation auc = 0.680282
Epoch 261
Validation auc = 0.693427
Epoch 262
Validation auc = 0.682160
Epoch 263
Validation auc = 0.662441
Epoch 264
Loss = 2.0116e-02, PNorm = 42.4867, GNorm = 1.5206, lr_0 = 6.4376e-04
Validation auc = 0.654460
Epoch 265
Validation auc = 0.648357
Epoch 266
Validation auc = 0.646009
Epoch 267
Validation auc = 0.643192
Epoch 268
Validation auc = 0.639906
Epoch 269
Loss = 1.6825e-02, PNorm = 42.5570, GNorm = 0.6128, lr_0 = 6.3826e-04
Validation auc = 0.645070
Epoch 270
Validation auc = 0.658216
Epoch 271
Validation auc = 0.667136
Epoch 272
Validation auc = 0.667136
Epoch 273
Validation auc = 0.670423
Epoch 274
Loss = 3.7900e-03, PNorm = 42.6529, GNorm = 0.2432, lr_0 = 6.3280e-04
Validation auc = 0.673709
Epoch 275
Validation auc = 0.675587
Epoch 276
Validation auc = 0.676526
Epoch 277
Validation auc = 0.678404
Epoch 278
Validation auc = 0.677934
Epoch 279
Loss = 2.2706e-02, PNorm = 42.7250, GNorm = 0.7043, lr_0 = 6.2739e-04
Validation auc = 0.674178
Epoch 280
Validation auc = 0.665728
Epoch 281
Validation auc = 0.650704
Epoch 282
Validation auc = 0.654930
Epoch 283
Validation auc = 0.672300
Epoch 284
Loss = 6.6715e-03, PNorm = 42.8091, GNorm = 0.2309, lr_0 = 6.2203e-04
Validation auc = 0.679812
Epoch 285
Validation auc = 0.681221
Epoch 286
Validation auc = 0.682160
Epoch 287
Validation auc = 0.678873
Epoch 288
Validation auc = 0.675587
Epoch 289
Loss = 2.0174e-02, PNorm = 42.8948, GNorm = 0.1204, lr_0 = 6.1671e-04
Validation auc = 0.671831
Epoch 290
Validation auc = 0.672770
Epoch 291
Validation auc = 0.671831
Epoch 292
Validation auc = 0.671831
Epoch 293
Validation auc = 0.673709
Epoch 294
Loss = 1.2744e-02, PNorm = 42.9446, GNorm = 0.4630, lr_0 = 6.1144e-04
Validation auc = 0.675117
Epoch 295
Validation auc = 0.675117
Epoch 296
Validation auc = 0.673239
Epoch 297
Validation auc = 0.674178
Epoch 298
Validation auc = 0.674648
Epoch 299
Loss = 5.6973e-03, PNorm = 42.9917, GNorm = 0.1069, lr_0 = 6.0621e-04
Validation auc = 0.674648
Model 0 best validation auc = 0.721596 on epoch 4
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test auc = 0.783865
Ensemble test auc = 0.783865
Fold 3
Splitting data with seed 3
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.05, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=300, bias=False)
        (W_h): Linear(in_features=300, out_features=300, bias=False)
        (W_o): Linear(in_features=433, out_features=300, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.05, inplace=False)
    (1): Linear(in_features=2200, out_features=300, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.05, inplace=False)
    (4): Linear(in_features=300, out_features=1, bias=True)
  )
)
Number of parameters = 924,901
Moving model to cuda
Epoch 0
Validation auc = 0.667136
Epoch 1
Validation auc = 0.736620
Epoch 2
Validation auc = 0.694836
Epoch 3
Validation auc = 0.692958
Epoch 4
Loss = 7.0410e-01, PNorm = 37.1939, GNorm = 11.5334, lr_0 = 6.5000e-04
Validation auc = 0.684507
Epoch 5
Validation auc = 0.666667
Epoch 6
Validation auc = 0.663380
Epoch 7
Validation auc = 0.651174
Epoch 8
Validation auc = 0.636620
Epoch 9
Loss = 5.4851e-01, PNorm = 37.2566, GNorm = 3.2231, lr_0 = 9.9743e-04
Validation auc = 0.632864
Epoch 10
Validation auc = 0.608451
Epoch 11
Validation auc = 0.619718
Epoch 12
Validation auc = 0.607512
Epoch 13
Validation auc = 0.613146
Epoch 14
Loss = 4.3846e-01, PNorm = 37.3594, GNorm = 8.7488, lr_0 = 9.8890e-04
Validation auc = 0.616432
Epoch 15
Validation auc = 0.629577
Epoch 16
Validation auc = 0.638498
Epoch 17
Validation auc = 0.644601
Epoch 18
Validation auc = 0.638028
Epoch 19
Loss = 4.6006e-01, PNorm = 37.4875, GNorm = 6.4862, lr_0 = 9.8045e-04
Validation auc = 0.638967
Epoch 20
Validation auc = 0.635681
Epoch 21
Validation auc = 0.630516
Epoch 22
Validation auc = 0.628169
Epoch 23
Validation auc = 0.620188
Epoch 24
Loss = 3.9940e-01, PNorm = 37.6440, GNorm = 6.6520, lr_0 = 9.7207e-04
Validation auc = 0.622535
Epoch 25
Validation auc = 0.615962
Epoch 26
Validation auc = 0.623944
Epoch 27
Validation auc = 0.608451
Epoch 28
Validation auc = 0.624883
Epoch 29
Loss = 2.8567e-01, PNorm = 37.8039, GNorm = 5.9872, lr_0 = 9.6376e-04
Validation auc = 0.630047
Epoch 30
Validation auc = 0.640845
Epoch 31
Validation auc = 0.639437
Epoch 32
Validation auc = 0.647887
Epoch 33
Validation auc = 0.622535
Epoch 34
Loss = 2.9574e-01, PNorm = 37.9838, GNorm = 1.5026, lr_0 = 9.5552e-04
Validation auc = 0.646479
Epoch 35
Validation auc = 0.631455
Epoch 36
Validation auc = 0.607042
Epoch 37
Validation auc = 0.630047
Epoch 38
Validation auc = 0.624413
Epoch 39
Loss = 2.7282e-01, PNorm = 38.1726, GNorm = 2.8406, lr_0 = 9.4735e-04
Validation auc = 0.628169
Epoch 40
Validation auc = 0.635211
Epoch 41
Validation auc = 0.615962
Epoch 42
Validation auc = 0.634272
Epoch 43
Validation auc = 0.623005
Epoch 44
Loss = 2.6766e-01, PNorm = 38.3474, GNorm = 7.2671, lr_0 = 9.3925e-04
Validation auc = 0.597653
Epoch 45
Validation auc = 0.613615
Epoch 46
Validation auc = 0.619718
Epoch 47
Validation auc = 0.630047
Epoch 48
Validation auc = 0.631455
Epoch 49
Loss = 2.3180e-01, PNorm = 38.5469, GNorm = 3.0003, lr_0 = 9.3122e-04
Validation auc = 0.630986
Epoch 50
Validation auc = 0.630516
Epoch 51
Validation auc = 0.626761
Epoch 52
Validation auc = 0.592019
Epoch 53
Validation auc = 0.615962
Epoch 54
Loss = 2.1631e-01, PNorm = 38.7307, GNorm = 5.8714, lr_0 = 9.2326e-04
Validation auc = 0.615023
Epoch 55
Validation auc = 0.580282
Epoch 56
Validation auc = 0.612676
Epoch 57
Validation auc = 0.617840
Epoch 58
Validation auc = 0.620657
Epoch 59
Loss = 1.9043e-01, PNorm = 38.9224, GNorm = 1.6710, lr_0 = 9.1537e-04
Validation auc = 0.631455
Epoch 60
Validation auc = 0.634272
Epoch 61
Validation auc = 0.636620
Epoch 62
Validation auc = 0.618310
Epoch 63
Validation auc = 0.619718
Epoch 64
Loss = 1.0820e-01, PNorm = 39.0998, GNorm = 1.8624, lr_0 = 9.0754e-04
Validation auc = 0.613146
Epoch 65
Validation auc = 0.615493
Epoch 66
Validation auc = 0.619249
Epoch 67
Validation auc = 0.620657
Epoch 68
Validation auc = 0.613615
Epoch 69
Loss = 1.4307e-01, PNorm = 39.2646, GNorm = 2.7710, lr_0 = 8.9978e-04
Validation auc = 0.629577
Epoch 70
Validation auc = 0.638967
Epoch 71
Validation auc = 0.639906
Epoch 72
Validation auc = 0.643192
Epoch 73
Validation auc = 0.649765
Epoch 74
Loss = 7.2171e-02, PNorm = 39.4346, GNorm = 1.0088, lr_0 = 8.9209e-04
Validation auc = 0.651174
Epoch 75
Validation auc = 0.648826
Epoch 76
Validation auc = 0.646479
Epoch 77
Validation auc = 0.639906
Epoch 78
Validation auc = 0.632864
Epoch 79
Loss = 6.3074e-02, PNorm = 39.5870, GNorm = 1.6931, lr_0 = 8.8447e-04
Validation auc = 0.615023
Epoch 80
Validation auc = 0.607042
Epoch 81
Validation auc = 0.621127
Epoch 82
Validation auc = 0.620188
Epoch 83
Validation auc = 0.650704
Epoch 84
Loss = 9.0354e-02, PNorm = 39.7461, GNorm = 1.5544, lr_0 = 8.7691e-04
Validation auc = 0.651174
Epoch 85
Validation auc = 0.645540
Epoch 86
Validation auc = 0.648826
Epoch 87
Validation auc = 0.628169
Epoch 88
Validation auc = 0.592488
Epoch 89
Loss = 1.6936e-01, PNorm = 39.8945, GNorm = 3.5814, lr_0 = 8.6941e-04
Validation auc = 0.610329
Epoch 90
Validation auc = 0.615493
Epoch 91
Validation auc = 0.598122
Epoch 92
Validation auc = 0.608451
Epoch 93
Validation auc = 0.608451
Epoch 94
Loss = 1.0098e-01, PNorm = 40.0890, GNorm = 3.1859, lr_0 = 8.6198e-04
Validation auc = 0.618779
Epoch 95
Validation auc = 0.629577
Epoch 96
Validation auc = 0.635211
Epoch 97
Validation auc = 0.647887
Epoch 98
Validation auc = 0.653052
Epoch 99
Loss = 6.4735e-02, PNorm = 40.2567, GNorm = 1.9597, lr_0 = 8.5461e-04
Validation auc = 0.654460
Epoch 100
Validation auc = 0.653521
Epoch 101
Validation auc = 0.653991
Epoch 102
Validation auc = 0.641784
Epoch 103
Validation auc = 0.634742
Epoch 104
Loss = 5.8346e-02, PNorm = 40.4084, GNorm = 0.6354, lr_0 = 8.4730e-04
Validation auc = 0.648357
Epoch 105
Validation auc = 0.644601
Epoch 106
Validation auc = 0.636150
Epoch 107
Validation auc = 0.641784
Epoch 108
Validation auc = 0.646479
Epoch 109
Loss = 2.9374e-02, PNorm = 40.5442, GNorm = 1.0161, lr_0 = 8.4006e-04
Validation auc = 0.650235
Epoch 110
Validation auc = 0.656338
Epoch 111
Validation auc = 0.658216
Epoch 112
Validation auc = 0.668075
Epoch 113
Validation auc = 0.661972
Epoch 114
Loss = 4.9379e-02, PNorm = 40.6621, GNorm = 1.4745, lr_0 = 8.3288e-04
Validation auc = 0.646479
Epoch 115
Validation auc = 0.666197
Epoch 116
Validation auc = 0.651643
Epoch 117
Validation auc = 0.634272
Epoch 118
Validation auc = 0.625822
Epoch 119
Loss = 8.6577e-02, PNorm = 40.8056, GNorm = 0.6909, lr_0 = 8.2576e-04
Validation auc = 0.625822
Epoch 120
Validation auc = 0.639437
Epoch 121
Validation auc = 0.648357
Epoch 122
Validation auc = 0.648826
Epoch 123
Validation auc = 0.653052
Epoch 124
Loss = 6.3158e-02, PNorm = 40.9288, GNorm = 0.6047, lr_0 = 8.1870e-04
Validation auc = 0.655869
Epoch 125
Validation auc = 0.656808
Epoch 126
Validation auc = 0.654930
Epoch 127
Validation auc = 0.652582
Epoch 128
Validation auc = 0.650235
Epoch 129
Loss = 5.6259e-02, PNorm = 41.0142, GNorm = 1.4639, lr_0 = 8.1170e-04
Validation auc = 0.656338
Epoch 130
Validation auc = 0.663850
Epoch 131
Validation auc = 0.649296
Epoch 132
Validation auc = 0.635681
Epoch 133
Validation auc = 0.634272
Epoch 134
Loss = 5.8835e-02, PNorm = 41.1101, GNorm = 1.8909, lr_0 = 8.0476e-04
Validation auc = 0.645070
Epoch 135
Validation auc = 0.647887
Epoch 136
Validation auc = 0.654460
Epoch 137
Validation auc = 0.660563
Epoch 138
Validation auc = 0.681221
Epoch 139
Loss = 3.3880e-02, PNorm = 41.2364, GNorm = 1.0348, lr_0 = 7.9788e-04
Validation auc = 0.683568
Epoch 140
Validation auc = 0.677465
Epoch 141
Validation auc = 0.670892
Epoch 142
Validation auc = 0.673709
Epoch 143
Validation auc = 0.671831
Epoch 144
Loss = 3.8846e-02, PNorm = 41.3356, GNorm = 1.4887, lr_0 = 7.9106e-04
Validation auc = 0.669014
Epoch 145
Validation auc = 0.663850
Epoch 146
Validation auc = 0.666197
Epoch 147
Validation auc = 0.664789
Epoch 148
Validation auc = 0.662441
Epoch 149
Loss = 8.6080e-03, PNorm = 41.4163, GNorm = 0.1545, lr_0 = 7.8430e-04
Validation auc = 0.661502
Epoch 150
Validation auc = 0.660094
Epoch 151
Validation auc = 0.660563
Epoch 152
Validation auc = 0.658685
Epoch 153
Validation auc = 0.662911
Epoch 154
Loss = 1.0367e-02, PNorm = 41.4824, GNorm = 0.4200, lr_0 = 7.7759e-04
Validation auc = 0.664789
Epoch 155
Validation auc = 0.669014
Epoch 156
Validation auc = 0.669953
Epoch 157
Validation auc = 0.670892
Epoch 158
Validation auc = 0.673239
Epoch 159
Loss = 1.6710e-02, PNorm = 41.5388, GNorm = 0.3781, lr_0 = 7.7095e-04
Validation auc = 0.675117
Epoch 160
Validation auc = 0.677465
Epoch 161
Validation auc = 0.679343
Epoch 162
Validation auc = 0.680751
Epoch 163
Validation auc = 0.680282
Epoch 164
Loss = 1.3175e-02, PNorm = 41.5949, GNorm = 0.6220, lr_0 = 7.6436e-04
Validation auc = 0.680282
Epoch 165
Validation auc = 0.689202
Epoch 166
Validation auc = 0.684977
Epoch 167
Validation auc = 0.685915
Epoch 168
Validation auc = 0.684507
Epoch 169
Loss = 6.5720e-03, PNorm = 41.6466, GNorm = 0.0589, lr_0 = 7.5782e-04
Validation auc = 0.683568
Epoch 170
Validation auc = 0.684507
Epoch 171
Validation auc = 0.682629
Epoch 172
Validation auc = 0.682629
Epoch 173
Validation auc = 0.684507
Epoch 174
Loss = 1.3186e-02, PNorm = 41.6932, GNorm = 0.6386, lr_0 = 7.5134e-04
Validation auc = 0.683568
Epoch 175
Validation auc = 0.684038
Epoch 176
Validation auc = 0.682629
Epoch 177
Validation auc = 0.681690
Epoch 178
Validation auc = 0.678404
Epoch 179
Loss = 1.5095e-02, PNorm = 41.7370, GNorm = 0.5600, lr_0 = 7.4492e-04
Validation auc = 0.678404
Epoch 180
Validation auc = 0.677934
Epoch 181
Validation auc = 0.671831
Epoch 182
Validation auc = 0.669484
Epoch 183
Validation auc = 0.670423
Epoch 184
Loss = 4.8657e-03, PNorm = 41.7769, GNorm = 0.1208, lr_0 = 7.3855e-04
Validation auc = 0.677465
Epoch 185
Validation auc = 0.670892
Epoch 186
Validation auc = 0.663380
Epoch 187
Validation auc = 0.665258
Epoch 188
Validation auc = 0.671831
Epoch 189
Loss = 7.0687e-03, PNorm = 41.8193, GNorm = 0.1304, lr_0 = 7.3224e-04
Validation auc = 0.678873
Epoch 190
Validation auc = 0.684507
Epoch 191
Validation auc = 0.685446
Epoch 192
Validation auc = 0.686385
Epoch 193
Validation auc = 0.687324
Epoch 194
Loss = 3.6026e-03, PNorm = 41.8612, GNorm = 0.0855, lr_0 = 7.2598e-04
Validation auc = 0.688732
Epoch 195
Validation auc = 0.688263
Epoch 196
Validation auc = 0.689202
Epoch 197
Validation auc = 0.690610
Epoch 198
Validation auc = 0.684977
Epoch 199
Loss = 5.0497e-03, PNorm = 41.9058, GNorm = 0.1489, lr_0 = 7.1977e-04
Validation auc = 0.678873
Epoch 200
Validation auc = 0.669484
Epoch 201
Validation auc = 0.663850
Epoch 202
Validation auc = 0.666667
Epoch 203
Validation auc = 0.667606
Epoch 204
Loss = 3.9239e-03, PNorm = 41.9443, GNorm = 0.0543, lr_0 = 7.1362e-04
Validation auc = 0.666197
Epoch 205
Validation auc = 0.666667
Epoch 206
Validation auc = 0.664789
Epoch 207
Validation auc = 0.666667
Epoch 208
Validation auc = 0.669484
Epoch 209
Loss = 8.2954e-03, PNorm = 41.9772, GNorm = 0.2390, lr_0 = 7.0752e-04
Validation auc = 0.670892
Epoch 210
Validation auc = 0.670892
Epoch 211
Validation auc = 0.669953
Epoch 212
Validation auc = 0.667136
Epoch 213
Validation auc = 0.669953
Epoch 214
Loss = 6.3259e-03, PNorm = 42.0235, GNorm = 0.0602, lr_0 = 7.0147e-04
Validation auc = 0.672300
Epoch 215
Validation auc = 0.673239
Epoch 216
Validation auc = 0.673709
Epoch 217
Validation auc = 0.675587
Epoch 218
Validation auc = 0.677934
Epoch 219
Loss = 2.2375e-03, PNorm = 42.0601, GNorm = 0.0596, lr_0 = 6.9548e-04
Validation auc = 0.678873
Epoch 220
Validation auc = 0.681221
Epoch 221
Validation auc = 0.681690
Epoch 222
Validation auc = 0.678873
Epoch 223
Validation auc = 0.678404
Epoch 224
Loss = 2.6192e-03, PNorm = 42.0902, GNorm = 0.1153, lr_0 = 6.8953e-04
Validation auc = 0.679343
Epoch 225
Validation auc = 0.677934
Epoch 226
Validation auc = 0.675117
Epoch 227
Validation auc = 0.675587
Epoch 228
Validation auc = 0.675587
Epoch 229
Loss = 2.5558e-03, PNorm = 42.1185, GNorm = 0.0718, lr_0 = 6.8364e-04
Validation auc = 0.676995
Epoch 230
Validation auc = 0.678873
Epoch 231
Validation auc = 0.677465
Epoch 232
Validation auc = 0.677465
Epoch 233
Validation auc = 0.675587
Epoch 234
Loss = 3.8147e-03, PNorm = 42.1442, GNorm = 0.0403, lr_0 = 6.7779e-04
Validation auc = 0.675587
Epoch 235
Validation auc = 0.673239
Epoch 236
Validation auc = 0.674178
Epoch 237
Validation auc = 0.675117
Epoch 238
Validation auc = 0.675587
Epoch 239
Loss = 2.0615e-03, PNorm = 42.1711, GNorm = 0.1482, lr_0 = 6.7200e-04
Validation auc = 0.676995
Epoch 240
Validation auc = 0.676526
Epoch 241
Validation auc = 0.677934
Epoch 242
Validation auc = 0.676056
Epoch 243
Validation auc = 0.676056
Epoch 244
Loss = 3.3048e-03, PNorm = 42.1993, GNorm = 0.0575, lr_0 = 6.6625e-04
Validation auc = 0.677465
Epoch 245
Validation auc = 0.680282
Epoch 246
Validation auc = 0.680282
Epoch 247
Validation auc = 0.679812
Epoch 248
Validation auc = 0.677934
Epoch 249
Loss = 1.3780e-03, PNorm = 42.2258, GNorm = 0.0368, lr_0 = 6.6056e-04
Validation auc = 0.678873
Epoch 250
Validation auc = 0.678873
Epoch 251
Validation auc = 0.682160
Epoch 252
Validation auc = 0.681690
Epoch 253
Validation auc = 0.683568
Epoch 254
Loss = 3.4358e-03, PNorm = 42.2480, GNorm = 0.1652, lr_0 = 6.5491e-04
Validation auc = 0.684977
Epoch 255
Validation auc = 0.684038
Epoch 256
Validation auc = 0.682629
Epoch 257
Validation auc = 0.679343
Epoch 258
Validation auc = 0.682629
Epoch 259
Loss = 1.8843e-03, PNorm = 42.2730, GNorm = 0.0207, lr_0 = 6.4931e-04
Validation auc = 0.684038
Epoch 260
Validation auc = 0.683099
Epoch 261
Validation auc = 0.682160
Epoch 262
Validation auc = 0.680282
Epoch 263
Validation auc = 0.677465
Epoch 264
Loss = 9.8491e-03, PNorm = 42.3013, GNorm = 0.0475, lr_0 = 6.4376e-04
Validation auc = 0.683099
Epoch 265
Validation auc = 0.680751
Epoch 266
Validation auc = 0.681221
Epoch 267
Validation auc = 0.682629
Epoch 268
Validation auc = 0.682629
Epoch 269
Loss = 4.0807e-03, PNorm = 42.3355, GNorm = 0.0891, lr_0 = 6.3826e-04
Validation auc = 0.682160
Epoch 270
Validation auc = 0.679812
Epoch 271
Validation auc = 0.683099
Epoch 272
Validation auc = 0.683568
Epoch 273
Validation auc = 0.684038
Epoch 274
Loss = 1.4838e-03, PNorm = 42.3664, GNorm = 0.0752, lr_0 = 6.3280e-04
Validation auc = 0.684038
Epoch 275
Validation auc = 0.685915
Epoch 276
Validation auc = 0.685915
Epoch 277
Validation auc = 0.686854
Epoch 278
Validation auc = 0.684977
Epoch 279
Loss = 1.3178e-03, PNorm = 42.3862, GNorm = 0.0303, lr_0 = 6.2739e-04
Validation auc = 0.685446
Epoch 280
Validation auc = 0.684507
Epoch 281
Validation auc = 0.682160
Epoch 282
Validation auc = 0.682160
Epoch 283
Validation auc = 0.681221
Epoch 284
Loss = 2.9928e-03, PNorm = 42.4029, GNorm = 0.0860, lr_0 = 6.2203e-04
Validation auc = 0.680282
Epoch 285
Validation auc = 0.681690
Epoch 286
Validation auc = 0.682160
Epoch 287
Validation auc = 0.681690
Epoch 288
Validation auc = 0.681221
Epoch 289
Loss = 1.0398e-03, PNorm = 42.4179, GNorm = 0.0193, lr_0 = 6.1671e-04
Validation auc = 0.679812
Epoch 290
Validation auc = 0.680282
Epoch 291
Validation auc = 0.680282
Epoch 292
Validation auc = 0.676526
Epoch 293
Validation auc = 0.678873
Epoch 294
Loss = 6.4099e-04, PNorm = 42.4398, GNorm = 0.0118, lr_0 = 6.1144e-04
Validation auc = 0.677465
Epoch 295
Validation auc = 0.678873
Epoch 296
Validation auc = 0.678873
Epoch 297
Validation auc = 0.677465
Epoch 298
Validation auc = 0.680282
Epoch 299
Loss = 1.7396e-03, PNorm = 42.4605, GNorm = 0.1049, lr_0 = 6.0621e-04
Validation auc = 0.682160
Model 0 best validation auc = 0.736620 on epoch 1
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test auc = 0.738712
Ensemble test auc = 0.738712
Fold 4
Splitting data with seed 4
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.05, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=300, bias=False)
        (W_h): Linear(in_features=300, out_features=300, bias=False)
        (W_o): Linear(in_features=433, out_features=300, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.05, inplace=False)
    (1): Linear(in_features=2200, out_features=300, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.05, inplace=False)
    (4): Linear(in_features=300, out_features=1, bias=True)
  )
)
Number of parameters = 924,901
Moving model to cuda
Epoch 0
Validation auc = 0.663380
Epoch 1
Validation auc = 0.708920
Epoch 2
Validation auc = 0.666667
Epoch 3
Validation auc = 0.672300
Epoch 4
Loss = 6.1766e-01, PNorm = 37.1925, GNorm = 5.2109, lr_0 = 6.5000e-04
Validation auc = 0.721596
Epoch 5
Validation auc = 0.727700
Epoch 6
Validation auc = 0.668545
Epoch 7
Validation auc = 0.665258
Epoch 8
Validation auc = 0.676526
Epoch 9
Loss = 5.7059e-01, PNorm = 37.2471, GNorm = 2.6731, lr_0 = 9.9743e-04
Validation auc = 0.635211
Epoch 10
Validation auc = 0.640845
Epoch 11
Validation auc = 0.584507
Epoch 12
Validation auc = 0.641315
Epoch 13
Validation auc = 0.630047
Epoch 14
Loss = 4.7480e-01, PNorm = 37.3483, GNorm = 4.6767, lr_0 = 9.8890e-04
Validation auc = 0.612676
Epoch 15
Validation auc = 0.614085
Epoch 16
Validation auc = 0.619249
Epoch 17
Validation auc = 0.624883
Epoch 18
Validation auc = 0.642254
Epoch 19
Loss = 5.3368e-01, PNorm = 37.4707, GNorm = 3.1092, lr_0 = 9.8045e-04
Validation auc = 0.637559
Epoch 20
Validation auc = 0.639906
Epoch 21
Validation auc = 0.609390
Epoch 22
Validation auc = 0.641315
Epoch 23
Validation auc = 0.641315
Epoch 24
Loss = 3.5326e-01, PNorm = 37.6149, GNorm = 1.9206, lr_0 = 9.7207e-04
Validation auc = 0.638028
Epoch 25
Validation auc = 0.638967
Epoch 26
Validation auc = 0.626291
Epoch 27
Validation auc = 0.611268
Epoch 28
Validation auc = 0.618310
Epoch 29
Loss = 3.4434e-01, PNorm = 37.7846, GNorm = 4.5570, lr_0 = 9.6376e-04
Validation auc = 0.607981
Epoch 30
Validation auc = 0.616432
Epoch 31
Validation auc = 0.618779
Epoch 32
Validation auc = 0.608920
Epoch 33
Validation auc = 0.624883
Epoch 34
Loss = 4.1947e-01, PNorm = 37.9702, GNorm = 8.6857, lr_0 = 9.5552e-04
Validation auc = 0.627230
Epoch 35
Validation auc = 0.606573
Epoch 36
Validation auc = 0.620657
Epoch 37
Validation auc = 0.622535
Epoch 38
Validation auc = 0.608920
Epoch 39
Loss = 3.1738e-01, PNorm = 38.1538, GNorm = 1.5501, lr_0 = 9.4735e-04
Validation auc = 0.598592
Epoch 40
Validation auc = 0.615962
Epoch 41
Validation auc = 0.622066
Epoch 42
Validation auc = 0.622535
Epoch 43
Validation auc = 0.622066
Epoch 44
Loss = 1.8955e-01, PNorm = 38.3515, GNorm = 1.1186, lr_0 = 9.3925e-04
Validation auc = 0.617371
Epoch 45
Validation auc = 0.615493
Epoch 46
Validation auc = 0.622535
Epoch 47
Validation auc = 0.615962
Epoch 48
Validation auc = 0.596714
Epoch 49
Loss = 2.4326e-01, PNorm = 38.5570, GNorm = 2.9991, lr_0 = 9.3122e-04
Validation auc = 0.621127
Epoch 50
Validation auc = 0.614554
Epoch 51
Validation auc = 0.643662
Epoch 52
Validation auc = 0.629577
Epoch 53
Validation auc = 0.634742
Epoch 54
Loss = 1.9648e-01, PNorm = 38.7644, GNorm = 7.2243, lr_0 = 9.2326e-04
Validation auc = 0.634272
Epoch 55
Validation auc = 0.622535
Epoch 56
Validation auc = 0.632864
Epoch 57
Validation auc = 0.633333
Epoch 58
Validation auc = 0.638967
Epoch 59
Loss = 3.9084e-01, PNorm = 38.9794, GNorm = 5.8207, lr_0 = 9.1537e-04
Validation auc = 0.634272
Epoch 60
Validation auc = 0.630986
Epoch 61
Validation auc = 0.627700
Epoch 62
Validation auc = 0.607042
Epoch 63
Validation auc = 0.626291
Epoch 64
Loss = 1.6171e-01, PNorm = 39.1741, GNorm = 5.2572, lr_0 = 9.0754e-04
Validation auc = 0.627230
Epoch 65
Validation auc = 0.615023
Epoch 66
Validation auc = 0.623005
Epoch 67
Validation auc = 0.642723
Epoch 68
Validation auc = 0.649765
Epoch 69
Loss = 9.4152e-02, PNorm = 39.3746, GNorm = 0.6414, lr_0 = 8.9978e-04
Validation auc = 0.650235
Epoch 70
Validation auc = 0.652582
Epoch 71
Validation auc = 0.653521
Epoch 72
Validation auc = 0.636150
Epoch 73
Validation auc = 0.641315
Epoch 74
Loss = 1.0699e-01, PNorm = 39.5525, GNorm = 2.5665, lr_0 = 8.9209e-04
Validation auc = 0.648357
Epoch 75
Validation auc = 0.629577
Epoch 76
Validation auc = 0.634272
Epoch 77
Validation auc = 0.646009
Epoch 78
Validation auc = 0.656808
Epoch 79
Loss = 9.5259e-02, PNorm = 39.7212, GNorm = 2.8429, lr_0 = 8.8447e-04
Validation auc = 0.657746
Epoch 80
Validation auc = 0.653991
Epoch 81
Validation auc = 0.645070
Epoch 82
Validation auc = 0.631455
Epoch 83
Validation auc = 0.626761
Epoch 84
Loss = 1.1042e-01, PNorm = 39.8721, GNorm = 3.0968, lr_0 = 8.7691e-04
Validation auc = 0.633333
Epoch 85
Validation auc = 0.637559
Epoch 86
Validation auc = 0.645540
Epoch 87
Validation auc = 0.650235
Epoch 88
Validation auc = 0.648826
Epoch 89
Loss = 5.2111e-02, PNorm = 40.0182, GNorm = 1.7822, lr_0 = 8.6941e-04
Validation auc = 0.645540
Epoch 90
Validation auc = 0.651643
Epoch 91
Validation auc = 0.653991
Epoch 92
Validation auc = 0.659624
Epoch 93
Validation auc = 0.658216
Epoch 94
Loss = 7.1769e-02, PNorm = 40.1475, GNorm = 0.7846, lr_0 = 8.6198e-04
Validation auc = 0.657277
Epoch 95
Validation auc = 0.661033
Epoch 96
Validation auc = 0.661033
Epoch 97
Validation auc = 0.659155
Epoch 98
Validation auc = 0.665258
Epoch 99
Loss = 3.9138e-02, PNorm = 40.2699, GNorm = 1.0178, lr_0 = 8.5461e-04
Validation auc = 0.663380
Epoch 100
Validation auc = 0.659155
Epoch 101
Validation auc = 0.659155
Epoch 102
Validation auc = 0.659624
Epoch 103
Validation auc = 0.668075
Epoch 104
Loss = 2.3644e-02, PNorm = 40.3854, GNorm = 0.4566, lr_0 = 8.4730e-04
Validation auc = 0.668545
Epoch 105
Validation auc = 0.670892
Epoch 106
Validation auc = 0.664789
Epoch 107
Validation auc = 0.657277
Epoch 108
Validation auc = 0.661033
Epoch 109
Loss = 5.2193e-02, PNorm = 40.4904, GNorm = 0.8460, lr_0 = 8.4006e-04
Validation auc = 0.653991
Epoch 110
Validation auc = 0.648826
Epoch 111
Validation auc = 0.654930
Epoch 112
Validation auc = 0.666667
Epoch 113
Validation auc = 0.657746
Epoch 114
Loss = 3.8271e-02, PNorm = 40.6127, GNorm = 1.4769, lr_0 = 8.3288e-04
Validation auc = 0.649296
Epoch 115
Validation auc = 0.650704
Epoch 116
Validation auc = 0.649765
Epoch 117
Validation auc = 0.653052
Epoch 118
Validation auc = 0.652582
Epoch 119
Loss = 1.2768e-01, PNorm = 40.7466, GNorm = 0.2767, lr_0 = 8.2576e-04
Validation auc = 0.669953
Epoch 120
Validation auc = 0.669014
Epoch 121
Validation auc = 0.646009
Epoch 122
Validation auc = 0.649296
Epoch 123
Validation auc = 0.654460
Epoch 124
Loss = 3.3324e-02, PNorm = 40.8792, GNorm = 0.6358, lr_0 = 8.1870e-04
Validation auc = 0.655399
Epoch 125
Validation auc = 0.655869
Epoch 126
Validation auc = 0.659624
Epoch 127
Validation auc = 0.659624
Epoch 128
Validation auc = 0.658216
Epoch 129
Loss = 2.1772e-02, PNorm = 40.9837, GNorm = 0.3442, lr_0 = 8.1170e-04
Validation auc = 0.660563
Epoch 130
Validation auc = 0.671362
Epoch 131
Validation auc = 0.678873
Epoch 132
Validation auc = 0.679812
Epoch 133
Validation auc = 0.671831
Epoch 134
Loss = 4.6687e-02, PNorm = 41.0816, GNorm = 0.2154, lr_0 = 8.0476e-04
Validation auc = 0.678404
Epoch 135
Validation auc = 0.677465
Epoch 136
Validation auc = 0.668545
Epoch 137
Validation auc = 0.669014
Epoch 138
Validation auc = 0.670423
Epoch 139
Loss = 4.6392e-02, PNorm = 41.1616, GNorm = 0.6187, lr_0 = 7.9788e-04
Validation auc = 0.677465
Epoch 140
Validation auc = 0.668545
Epoch 141
Validation auc = 0.657746
Epoch 142
Validation auc = 0.650235
Epoch 143
Validation auc = 0.652582
Epoch 144
Loss = 6.3496e-02, PNorm = 41.2802, GNorm = 1.6046, lr_0 = 7.9106e-04
Validation auc = 0.659624
Epoch 145
Validation auc = 0.664319
Epoch 146
Validation auc = 0.653991
Epoch 147
Validation auc = 0.661972
Epoch 148
Validation auc = 0.669014
Epoch 149
Loss = 2.5205e-02, PNorm = 41.4166, GNorm = 1.1429, lr_0 = 7.8430e-04
Validation auc = 0.672770
Epoch 150
Validation auc = 0.671831
Epoch 151
Validation auc = 0.673239
Epoch 152
Validation auc = 0.673709
Epoch 153
Validation auc = 0.678404
Epoch 154
Loss = 1.4713e-02, PNorm = 41.5180, GNorm = 0.1650, lr_0 = 7.7759e-04
Validation auc = 0.681221
Epoch 155
Validation auc = 0.678404
Epoch 156
Validation auc = 0.675117
Epoch 157
Validation auc = 0.676995
Epoch 158
Validation auc = 0.677934
Epoch 159
Loss = 9.6526e-03, PNorm = 41.5886, GNorm = 0.2287, lr_0 = 7.7095e-04
Validation auc = 0.676056
Epoch 160
Validation auc = 0.675117
Epoch 161
Validation auc = 0.670423
Epoch 162
Validation auc = 0.669014
Epoch 163
Validation auc = 0.675587
Epoch 164
Loss = 1.1901e-02, PNorm = 41.6483, GNorm = 0.6299, lr_0 = 7.6436e-04
Validation auc = 0.680751
Epoch 165
Validation auc = 0.683099
Epoch 166
Validation auc = 0.681221
Epoch 167
Validation auc = 0.674178
Epoch 168
Validation auc = 0.670423
Epoch 169
Loss = 7.2302e-03, PNorm = 41.7101, GNorm = 0.0708, lr_0 = 7.5782e-04
Validation auc = 0.674648
Epoch 170
Validation auc = 0.677934
Epoch 171
Validation auc = 0.685446
Epoch 172
Validation auc = 0.667136
Epoch 173
Validation auc = 0.661972
Epoch 174
Loss = 1.9751e-02, PNorm = 41.7610, GNorm = 0.3184, lr_0 = 7.5134e-04
Validation auc = 0.670423
Epoch 175
Validation auc = 0.679343
Epoch 176
Validation auc = 0.682629
Epoch 177
Validation auc = 0.679812
Epoch 178
Validation auc = 0.675587
Epoch 179
Loss = 5.5499e-03, PNorm = 41.8164, GNorm = 0.1828, lr_0 = 7.4492e-04
Validation auc = 0.673709
Epoch 180
Validation auc = 0.679343
Epoch 181
Validation auc = 0.678873
Epoch 182
Validation auc = 0.684038
Epoch 183
Validation auc = 0.678404
Epoch 184
Loss = 1.4982e-02, PNorm = 41.8806, GNorm = 0.5144, lr_0 = 7.3855e-04
Validation auc = 0.675117
Epoch 185
Validation auc = 0.675117
Epoch 186
Validation auc = 0.676056
Epoch 187
Validation auc = 0.676056
Epoch 188
Validation auc = 0.676526
Epoch 189
Loss = 2.8708e-03, PNorm = 41.9450, GNorm = 0.0422, lr_0 = 7.3224e-04
Validation auc = 0.675587
Epoch 190
Validation auc = 0.675117
Epoch 191
Validation auc = 0.676995
Epoch 192
Validation auc = 0.677465
Epoch 193
Validation auc = 0.680282
Epoch 194
Loss = 1.0252e-02, PNorm = 41.9883, GNorm = 0.2297, lr_0 = 7.2598e-04
Validation auc = 0.682629
Epoch 195
Validation auc = 0.683568
Epoch 196
Validation auc = 0.680751
Epoch 197
Validation auc = 0.676526
Epoch 198
Validation auc = 0.670892
Epoch 199
Loss = 1.7707e-03, PNorm = 42.0315, GNorm = 0.0436, lr_0 = 7.1977e-04
Validation auc = 0.672300
Epoch 200
Validation auc = 0.670892
Epoch 201
Validation auc = 0.678873
Epoch 202
Validation auc = 0.683099
Epoch 203
Validation auc = 0.687793
Epoch 204
Loss = 5.2502e-03, PNorm = 42.0652, GNorm = 0.1823, lr_0 = 7.1362e-04
Validation auc = 0.683568
Epoch 205
Validation auc = 0.679812
Epoch 206
Validation auc = 0.676526
Epoch 207
Validation auc = 0.676526
Epoch 208
Validation auc = 0.684038
Epoch 209
Loss = 4.9561e-03, PNorm = 42.0944, GNorm = 0.1817, lr_0 = 7.0752e-04
Validation auc = 0.686385
Epoch 210
Validation auc = 0.684038
Epoch 211
Validation auc = 0.678404
Epoch 212
Validation auc = 0.677934
Epoch 213
Validation auc = 0.679812
Epoch 214
Loss = 1.2026e-02, PNorm = 42.1268, GNorm = 0.1468, lr_0 = 7.0147e-04
Validation auc = 0.688732
Epoch 215
Validation auc = 0.694366
Epoch 216
Validation auc = 0.693427
Epoch 217
Validation auc = 0.683568
Epoch 218
Validation auc = 0.677465
Epoch 219
Loss = 4.1698e-03, PNorm = 42.1645, GNorm = 0.1699, lr_0 = 6.9548e-04
Validation auc = 0.674178
Epoch 220
Validation auc = 0.674648
Epoch 221
Validation auc = 0.676056
Epoch 222
Validation auc = 0.677465
Epoch 223
Validation auc = 0.680282
Epoch 224
Loss = 6.6154e-03, PNorm = 42.1944, GNorm = 0.1520, lr_0 = 6.8953e-04
Validation auc = 0.678404
Epoch 225
Validation auc = 0.676056
Epoch 226
Validation auc = 0.674178
Epoch 227
Validation auc = 0.671831
Epoch 228
Validation auc = 0.671831
Epoch 229
Loss = 4.9631e-03, PNorm = 42.2297, GNorm = 0.1000, lr_0 = 6.8364e-04
Validation auc = 0.672770
Epoch 230
Validation auc = 0.674648
Epoch 231
Validation auc = 0.678873
Epoch 232
Validation auc = 0.679812
Epoch 233
Validation auc = 0.680751
Epoch 234
Loss = 1.4012e-03, PNorm = 42.2589, GNorm = 0.0614, lr_0 = 6.7779e-04
Validation auc = 0.679343
Epoch 235
Validation auc = 0.684038
Epoch 236
Validation auc = 0.683568
Epoch 237
Validation auc = 0.678873
Epoch 238
Validation auc = 0.677934
Epoch 239
Loss = 9.2597e-04, PNorm = 42.2832, GNorm = 0.0277, lr_0 = 6.7200e-04
Validation auc = 0.676056
Epoch 240
Validation auc = 0.676526
Epoch 241
Validation auc = 0.679343
Epoch 242
Validation auc = 0.684038
Epoch 243
Validation auc = 0.688263
Epoch 244
Loss = 2.0266e-03, PNorm = 42.3023, GNorm = 0.0373, lr_0 = 6.6625e-04
Validation auc = 0.688732
Epoch 245
Validation auc = 0.693427
Epoch 246
Validation auc = 0.691549
Epoch 247
Validation auc = 0.684977
Epoch 248
Validation auc = 0.683099
Epoch 249
Loss = 3.2798e-03, PNorm = 42.3225, GNorm = 0.0902, lr_0 = 6.6056e-04
Validation auc = 0.679343
Epoch 250
Validation auc = 0.678404
Epoch 251
Validation auc = 0.680282
Epoch 252
Validation auc = 0.681690
Epoch 253
Validation auc = 0.682160
Epoch 254
Loss = 1.3325e-03, PNorm = 42.3398, GNorm = 0.0273, lr_0 = 6.5491e-04
Validation auc = 0.684507
Epoch 255
Validation auc = 0.683568
Epoch 256
Validation auc = 0.683568
Epoch 257
Validation auc = 0.682629
Epoch 258
Validation auc = 0.683568
Epoch 259
Loss = 8.0809e-04, PNorm = 42.3555, GNorm = 0.0311, lr_0 = 6.4931e-04
Validation auc = 0.684507
Epoch 260
Validation auc = 0.685915
Epoch 261
Validation auc = 0.686385
Epoch 262
Validation auc = 0.686854
Epoch 263
Validation auc = 0.688732
Epoch 264
Loss = 2.4235e-03, PNorm = 42.3709, GNorm = 0.0362, lr_0 = 6.4376e-04
Validation auc = 0.692019
Epoch 265
Validation auc = 0.693897
Epoch 266
Validation auc = 0.694366
Epoch 267
Validation auc = 0.693427
Epoch 268
Validation auc = 0.689671
Epoch 269
Loss = 1.3775e-03, PNorm = 42.3879, GNorm = 0.0267, lr_0 = 6.3826e-04
Validation auc = 0.687793
Epoch 270
Validation auc = 0.686854
Epoch 271
Validation auc = 0.686385
Epoch 272
Validation auc = 0.686385
Epoch 273
Validation auc = 0.687793
Epoch 274
Loss = 7.5926e-04, PNorm = 42.4036, GNorm = 0.0158, lr_0 = 6.3280e-04
Validation auc = 0.688732
Epoch 275
Validation auc = 0.688732
Epoch 276
Validation auc = 0.689671
Epoch 277
Validation auc = 0.690141
Epoch 278
Validation auc = 0.689671
Epoch 279
Loss = 1.1658e-03, PNorm = 42.4184, GNorm = 0.0450, lr_0 = 6.2739e-04
Validation auc = 0.687324
Epoch 280
Validation auc = 0.693897
Epoch 281
Validation auc = 0.691549
Epoch 282
Validation auc = 0.684977
Epoch 283
Validation auc = 0.681221
Epoch 284
Loss = 8.9986e-04, PNorm = 42.4378, GNorm = 0.0102, lr_0 = 6.2203e-04
Validation auc = 0.677465
Epoch 285
Validation auc = 0.673709
Epoch 286
Validation auc = 0.673239
Epoch 287
Validation auc = 0.675587
Epoch 288
Validation auc = 0.676995
Epoch 289
Loss = 8.7699e-04, PNorm = 42.4531, GNorm = 0.0319, lr_0 = 6.1671e-04
Validation auc = 0.684507
Epoch 290
Validation auc = 0.688263
Epoch 291
Validation auc = 0.691549
Epoch 292
Validation auc = 0.690610
Epoch 293
Validation auc = 0.691549
Epoch 294
Loss = 1.4160e-03, PNorm = 42.4679, GNorm = 0.0389, lr_0 = 6.1144e-04
Validation auc = 0.687793
Epoch 295
Validation auc = 0.686854
Epoch 296
Validation auc = 0.686385
Epoch 297
Validation auc = 0.688732
Epoch 298
Validation auc = 0.690610
Epoch 299
Loss = 5.9596e-04, PNorm = 42.4791, GNorm = 0.0153, lr_0 = 6.0621e-04
Validation auc = 0.691549
Model 0 best validation auc = 0.727700 on epoch 5
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test auc = 0.745334
Ensemble test auc = 0.745334
Fold 5
Splitting data with seed 5
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.05, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=300, bias=False)
        (W_h): Linear(in_features=300, out_features=300, bias=False)
        (W_o): Linear(in_features=433, out_features=300, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.05, inplace=False)
    (1): Linear(in_features=2200, out_features=300, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.05, inplace=False)
    (4): Linear(in_features=300, out_features=1, bias=True)
  )
)
Number of parameters = 924,901
Moving model to cuda
Epoch 0
Validation auc = 0.656808
Epoch 1
Validation auc = 0.746479
Epoch 2
Validation auc = 0.685446
Epoch 3
Validation auc = 0.722066
Epoch 4
Loss = 6.0632e-01, PNorm = 37.1973, GNorm = 2.6015, lr_0 = 6.5000e-04
Validation auc = 0.709859
Epoch 5
Validation auc = 0.674178
Epoch 6
Validation auc = 0.715023
Epoch 7
Validation auc = 0.653521
Epoch 8
Validation auc = 0.648826
Epoch 9
Loss = 4.6162e-01, PNorm = 37.2617, GNorm = 3.5499, lr_0 = 9.9743e-04
Validation auc = 0.647418
Epoch 10
Validation auc = 0.630516
Epoch 11
Validation auc = 0.645540
Epoch 12
Validation auc = 0.624413
Epoch 13
Validation auc = 0.619718
Epoch 14
Loss = 5.1052e-01, PNorm = 37.3599, GNorm = 1.8146, lr_0 = 9.8890e-04
Validation auc = 0.616432
Epoch 15
Validation auc = 0.639906
Epoch 16
Validation auc = 0.633803
Epoch 17
Validation auc = 0.630047
Epoch 18
Validation auc = 0.617840
Epoch 19
Loss = 5.1501e-01, PNorm = 37.4746, GNorm = 5.7823, lr_0 = 9.8045e-04
Validation auc = 0.629577
Epoch 20
Validation auc = 0.609390
Epoch 21
Validation auc = 0.585915
Epoch 22
Validation auc = 0.618310
Epoch 23
Validation auc = 0.604225
Epoch 24
Loss = 4.8527e-01, PNorm = 37.5958, GNorm = 11.4670, lr_0 = 9.7207e-04
Validation auc = 0.561502
Epoch 25
Validation auc = 0.601408
Epoch 26
Validation auc = 0.580282
Epoch 27
Validation auc = 0.592019
Epoch 28
Validation auc = 0.611737
Epoch 29
Loss = 3.6768e-01, PNorm = 37.7505, GNorm = 4.8395, lr_0 = 9.6376e-04
Validation auc = 0.598122
Epoch 30
Validation auc = 0.616901
Epoch 31
Validation auc = 0.618779
Epoch 32
Validation auc = 0.599531
Epoch 33
Validation auc = 0.608451
Epoch 34
Loss = 3.2196e-01, PNorm = 37.9057, GNorm = 2.7216, lr_0 = 9.5552e-04
Validation auc = 0.609859
Epoch 35
Validation auc = 0.583568
Epoch 36
Validation auc = 0.595305
Epoch 37
Validation auc = 0.571831
Epoch 38
Validation auc = 0.599061
Epoch 39
Loss = 3.0465e-01, PNorm = 38.1028, GNorm = 3.4765, lr_0 = 9.4735e-04
Validation auc = 0.611737
Epoch 40
Validation auc = 0.613615
Epoch 41
Validation auc = 0.623474
Epoch 42
Validation auc = 0.620657
Epoch 43
Validation auc = 0.624883
Epoch 44
Loss = 3.1943e-01, PNorm = 38.2943, GNorm = 2.8986, lr_0 = 9.3925e-04
Validation auc = 0.621127
Epoch 45
Validation auc = 0.607981
Epoch 46
Validation auc = 0.613615
Epoch 47
Validation auc = 0.605164
Epoch 48
Validation auc = 0.593897
Epoch 49
Loss = 2.4739e-01, PNorm = 38.4877, GNorm = 1.4310, lr_0 = 9.3122e-04
Validation auc = 0.610329
Epoch 50
Validation auc = 0.609390
Epoch 51
Validation auc = 0.612207
Epoch 52
Validation auc = 0.614554
Epoch 53
Validation auc = 0.611737
Epoch 54
Loss = 1.8285e-01, PNorm = 38.7005, GNorm = 2.1025, lr_0 = 9.2326e-04
Validation auc = 0.622535
Epoch 55
Validation auc = 0.595305
Epoch 56
Validation auc = 0.614085
Epoch 57
Validation auc = 0.615962
Epoch 58
Validation auc = 0.611268
Epoch 59
Loss = 1.2165e-01, PNorm = 38.9094, GNorm = 1.7286, lr_0 = 9.1537e-04
Validation auc = 0.617840
Epoch 60
Validation auc = 0.618310
Epoch 61
Validation auc = 0.619718
Epoch 62
Validation auc = 0.632864
Epoch 63
Validation auc = 0.627700
Epoch 64
Loss = 2.0096e-01, PNorm = 39.0897, GNorm = 12.0486, lr_0 = 9.0754e-04
Validation auc = 0.618310
Epoch 65
Validation auc = 0.620188
Epoch 66
Validation auc = 0.618310
Epoch 67
Validation auc = 0.611268
Epoch 68
Validation auc = 0.618310
Epoch 69
Loss = 1.5340e-01, PNorm = 39.2937, GNorm = 2.2530, lr_0 = 8.9978e-04
Validation auc = 0.602817
Epoch 70
Validation auc = 0.588732
Epoch 71
Validation auc = 0.623944
Epoch 72
Validation auc = 0.613146
Epoch 73
Validation auc = 0.598592
Epoch 74
Loss = 1.7457e-01, PNorm = 39.4728, GNorm = 2.2305, lr_0 = 8.9209e-04
Validation auc = 0.610798
Epoch 75
Validation auc = 0.618310
Epoch 76
Validation auc = 0.607512
Epoch 77
Validation auc = 0.598122
Epoch 78
Validation auc = 0.615493
Epoch 79
Loss = 6.7996e-02, PNorm = 39.6473, GNorm = 0.7582, lr_0 = 8.8447e-04
Validation auc = 0.625352
Epoch 80
Validation auc = 0.630047
Epoch 81
Validation auc = 0.630516
Epoch 82
Validation auc = 0.641315
Epoch 83
Validation auc = 0.645070
Epoch 84
Loss = 9.5625e-02, PNorm = 39.8016, GNorm = 1.6369, lr_0 = 8.7691e-04
Validation auc = 0.633333
Epoch 85
Validation auc = 0.631925
Epoch 86
Validation auc = 0.636620
Epoch 87
Validation auc = 0.635211
Epoch 88
Validation auc = 0.638028
Epoch 89
Loss = 1.0228e-01, PNorm = 39.9571, GNorm = 0.9970, lr_0 = 8.6941e-04
Validation auc = 0.645540
Epoch 90
Validation auc = 0.651174
Epoch 91
Validation auc = 0.649296
Epoch 92
Validation auc = 0.649765
Epoch 93
Validation auc = 0.651174
Epoch 94
Loss = 6.0278e-02, PNorm = 40.0996, GNorm = 2.1953, lr_0 = 8.6198e-04
Validation auc = 0.654930
Epoch 95
Validation auc = 0.659624
Epoch 96
Validation auc = 0.654460
Epoch 97
Validation auc = 0.642723
Epoch 98
Validation auc = 0.643662
Epoch 99
Loss = 6.0798e-02, PNorm = 40.2299, GNorm = 2.1979, lr_0 = 8.5461e-04
Validation auc = 0.632394
Epoch 100
Validation auc = 0.618779
Epoch 101
Validation auc = 0.624883
Epoch 102
Validation auc = 0.627700
Epoch 103
Validation auc = 0.627230
Epoch 104
Loss = 4.5073e-02, PNorm = 40.3435, GNorm = 0.5798, lr_0 = 8.4730e-04
Validation auc = 0.634742
Epoch 105
Validation auc = 0.635681
Epoch 106
Validation auc = 0.639906
Epoch 107
Validation auc = 0.641784
Epoch 108
Validation auc = 0.641315
Epoch 109
Loss = 2.7451e-02, PNorm = 40.4593, GNorm = 1.4689, lr_0 = 8.4006e-04
Validation auc = 0.640376
Epoch 110
Validation auc = 0.641315
Epoch 111
Validation auc = 0.647887
Epoch 112
Validation auc = 0.653521
Epoch 113
Validation auc = 0.654460
Epoch 114
Loss = 3.1185e-02, PNorm = 40.5571, GNorm = 0.3209, lr_0 = 8.3288e-04
Validation auc = 0.654460
Epoch 115
Validation auc = 0.654930
Epoch 116
Validation auc = 0.657746
Epoch 117
Validation auc = 0.660094
Epoch 118
Validation auc = 0.656338
Epoch 119
Loss = 4.0804e-02, PNorm = 40.6724, GNorm = 1.9272, lr_0 = 8.2576e-04
Validation auc = 0.655869
Epoch 120
Validation auc = 0.659624
Epoch 121
Validation auc = 0.654460
Epoch 122
Validation auc = 0.640845
Epoch 123
Validation auc = 0.641784
Epoch 124
Loss = 2.9376e-02, PNorm = 40.7915, GNorm = 0.1310, lr_0 = 8.1870e-04
Validation auc = 0.640845
Epoch 125
Validation auc = 0.641315
Epoch 126
Validation auc = 0.644601
Epoch 127
Validation auc = 0.641315
Epoch 128
Validation auc = 0.637089
Epoch 129
Loss = 4.3448e-02, PNorm = 40.9062, GNorm = 0.6338, lr_0 = 8.1170e-04
Validation auc = 0.644601
Epoch 130
Validation auc = 0.657277
Epoch 131
Validation auc = 0.641315
Epoch 132
Validation auc = 0.642254
Epoch 133
Validation auc = 0.650235
Epoch 134
Loss = 2.1355e-02, PNorm = 41.0345, GNorm = 0.8116, lr_0 = 8.0476e-04
Validation auc = 0.647887
Epoch 135
Validation auc = 0.642254
Epoch 136
Validation auc = 0.636150
Epoch 137
Validation auc = 0.647418
Epoch 138
Validation auc = 0.650704
Epoch 139
Loss = 2.5552e-02, PNorm = 41.1460, GNorm = 1.1479, lr_0 = 7.9788e-04
Validation auc = 0.643662
Epoch 140
Validation auc = 0.639437
Epoch 141
Validation auc = 0.626761
Epoch 142
Validation auc = 0.622066
Epoch 143
Validation auc = 0.623944
Epoch 144
Loss = 3.8931e-02, PNorm = 41.2322, GNorm = 0.8811, lr_0 = 7.9106e-04
Validation auc = 0.623005
Epoch 145
Validation auc = 0.621596
Epoch 146
Validation auc = 0.623474
Epoch 147
Validation auc = 0.625822
Epoch 148
Validation auc = 0.638967
Epoch 149
Loss = 3.3029e-02, PNorm = 41.3140, GNorm = 1.0921, lr_0 = 7.8430e-04
Validation auc = 0.644601
Epoch 150
Validation auc = 0.644131
Epoch 151
Validation auc = 0.638028
Epoch 152
Validation auc = 0.644131
Epoch 153
Validation auc = 0.651174
Epoch 154
Loss = 1.1502e-02, PNorm = 41.3929, GNorm = 0.2276, lr_0 = 7.7759e-04
Validation auc = 0.646479
Epoch 155
Validation auc = 0.641315
Epoch 156
Validation auc = 0.641784
Epoch 157
Validation auc = 0.645070
Epoch 158
Validation auc = 0.650235
Epoch 159
Loss = 9.4210e-03, PNorm = 41.4693, GNorm = 0.4337, lr_0 = 7.7095e-04
Validation auc = 0.654930
Epoch 160
Validation auc = 0.647887
Epoch 161
Validation auc = 0.646479
Epoch 162
Validation auc = 0.646948
Epoch 163
Validation auc = 0.654930
Epoch 164
Loss = 2.3422e-02, PNorm = 41.5263, GNorm = 0.4911, lr_0 = 7.6436e-04
Validation auc = 0.666667
Epoch 165
Validation auc = 0.658685
Epoch 166
Validation auc = 0.653052
Epoch 167
Validation auc = 0.650235
Epoch 168
Validation auc = 0.650704
Epoch 169
Loss = 1.1199e-02, PNorm = 41.5859, GNorm = 0.1192, lr_0 = 7.5782e-04
Validation auc = 0.649296
Epoch 170
Validation auc = 0.646009
Epoch 171
Validation auc = 0.644131
Epoch 172
Validation auc = 0.647418
Epoch 173
Validation auc = 0.650704
Epoch 174
Loss = 1.3952e-02, PNorm = 41.6376, GNorm = 0.5392, lr_0 = 7.5134e-04
Validation auc = 0.642254
Epoch 175
Validation auc = 0.632394
Epoch 176
Validation auc = 0.638498
Epoch 177
Validation auc = 0.648826
Epoch 178
Validation auc = 0.649296
Epoch 179
Loss = 8.1039e-03, PNorm = 41.7061, GNorm = 0.1078, lr_0 = 7.4492e-04
Validation auc = 0.653521
Epoch 180
Validation auc = 0.653052
Epoch 181
Validation auc = 0.655869
Epoch 182
Validation auc = 0.654460
Epoch 183
Validation auc = 0.655869
Epoch 184
Loss = 1.0814e-02, PNorm = 41.7586, GNorm = 0.5626, lr_0 = 7.3855e-04
Validation auc = 0.650704
Epoch 185
Validation auc = 0.643662
Epoch 186
Validation auc = 0.638967
Epoch 187
Validation auc = 0.644601
Epoch 188
Validation auc = 0.651174
Epoch 189
Loss = 9.6562e-03, PNorm = 41.8034, GNorm = 0.4246, lr_0 = 7.3224e-04
Validation auc = 0.652582
Epoch 190
Validation auc = 0.648357
Epoch 191
Validation auc = 0.648826
Epoch 192
Validation auc = 0.652582
Epoch 193
Validation auc = 0.652582
Epoch 194
Loss = 5.0298e-03, PNorm = 41.8519, GNorm = 0.0799, lr_0 = 7.2598e-04
Validation auc = 0.659624
Epoch 195
Validation auc = 0.661033
Epoch 196
Validation auc = 0.659624
Epoch 197
Validation auc = 0.655869
Epoch 198
Validation auc = 0.648826
Epoch 199
Loss = 1.1543e-02, PNorm = 41.8952, GNorm = 0.3414, lr_0 = 7.1977e-04
Validation auc = 0.653991
Epoch 200
Validation auc = 0.657746
Epoch 201
Validation auc = 0.663380
Epoch 202
Validation auc = 0.662441
Epoch 203
Validation auc = 0.659624
Epoch 204
Loss = 4.2924e-03, PNorm = 41.9294, GNorm = 0.1446, lr_0 = 7.1362e-04
Validation auc = 0.659155
Epoch 205
Validation auc = 0.655869
Epoch 206
Validation auc = 0.659624
Epoch 207
Validation auc = 0.660563
Epoch 208
Validation auc = 0.662911
Epoch 209
Loss = 5.0647e-03, PNorm = 41.9626, GNorm = 0.1600, lr_0 = 7.0752e-04
Validation auc = 0.660563
Epoch 210
Validation auc = 0.656338
Epoch 211
Validation auc = 0.653052
Epoch 212
Validation auc = 0.661502
Epoch 213
Validation auc = 0.663380
Epoch 214
Loss = 4.5553e-03, PNorm = 41.9989, GNorm = 0.2256, lr_0 = 7.0147e-04
Validation auc = 0.663380
Epoch 215
Validation auc = 0.662911
Epoch 216
Validation auc = 0.661972
Epoch 217
Validation auc = 0.658685
Epoch 218
Validation auc = 0.662911
Epoch 219
Loss = 4.8195e-03, PNorm = 42.0389, GNorm = 0.1141, lr_0 = 6.9548e-04
Validation auc = 0.663850
Epoch 220
Validation auc = 0.663380
Epoch 221
Validation auc = 0.657746
Epoch 222
Validation auc = 0.653521
Epoch 223
Validation auc = 0.651174
Epoch 224
Loss = 2.9969e-03, PNorm = 42.0693, GNorm = 0.0616, lr_0 = 6.8953e-04
Validation auc = 0.652113
Epoch 225
Validation auc = 0.653052
Epoch 226
Validation auc = 0.653991
Epoch 227
Validation auc = 0.655869
Epoch 228
Validation auc = 0.657277
Epoch 229
Loss = 2.1417e-03, PNorm = 42.0902, GNorm = 0.0543, lr_0 = 6.8364e-04
Validation auc = 0.655869
Epoch 230
Validation auc = 0.659155
Epoch 231
Validation auc = 0.661033
Epoch 232
Validation auc = 0.664319
Epoch 233
Validation auc = 0.668075
Epoch 234
Loss = 6.2737e-03, PNorm = 42.1113, GNorm = 0.3856, lr_0 = 6.7779e-04
Validation auc = 0.668075
Epoch 235
Validation auc = 0.663380
Epoch 236
Validation auc = 0.660094
Epoch 237
Validation auc = 0.660563
Epoch 238
Validation auc = 0.662911
Epoch 239
Loss = 3.0575e-03, PNorm = 42.1344, GNorm = 0.0450, lr_0 = 6.7200e-04
Validation auc = 0.666667
Epoch 240
Validation auc = 0.667136
Epoch 241
Validation auc = 0.668075
Epoch 242
Validation auc = 0.666667
Epoch 243
Validation auc = 0.665258
Epoch 244
Loss = 2.0285e-03, PNorm = 42.1592, GNorm = 0.0658, lr_0 = 6.6625e-04
Validation auc = 0.661033
Epoch 245
Validation auc = 0.661972
Epoch 246
Validation auc = 0.664319
Epoch 247
Validation auc = 0.666197
Epoch 248
Validation auc = 0.664789
Epoch 249
Loss = 5.5481e-03, PNorm = 42.1858, GNorm = 0.1735, lr_0 = 6.6056e-04
Validation auc = 0.658685
Epoch 250
Validation auc = 0.652113
Epoch 251
Validation auc = 0.648357
Epoch 252
Validation auc = 0.648357
Epoch 253
Validation auc = 0.653052
Epoch 254
Loss = 1.6891e-03, PNorm = 42.2138, GNorm = 0.1176, lr_0 = 6.5491e-04
Validation auc = 0.657277
Epoch 255
Validation auc = 0.656338
Epoch 256
Validation auc = 0.657277
Epoch 257
Validation auc = 0.656338
Epoch 258
Validation auc = 0.653521
Epoch 259
Loss = 1.4278e-03, PNorm = 42.2477, GNorm = 0.0414, lr_0 = 6.4931e-04
Validation auc = 0.650235
Epoch 260
Validation auc = 0.652582
Epoch 261
Validation auc = 0.657746
Epoch 262
Validation auc = 0.664789
Epoch 263
Validation auc = 0.667136
Epoch 264
Loss = 7.2600e-03, PNorm = 42.2763, GNorm = 0.2100, lr_0 = 6.4376e-04
Validation auc = 0.662911
Epoch 265
Validation auc = 0.658685
Epoch 266
Validation auc = 0.660094
Epoch 267
Validation auc = 0.662441
Epoch 268
Validation auc = 0.664319
Epoch 269
Loss = 2.4378e-03, PNorm = 42.3023, GNorm = 0.0628, lr_0 = 6.3826e-04
Validation auc = 0.665728
Epoch 270
Validation auc = 0.666197
Epoch 271
Validation auc = 0.665258
Epoch 272
Validation auc = 0.666197
Epoch 273
Validation auc = 0.665728
Epoch 274
Loss = 1.8163e-03, PNorm = 42.3252, GNorm = 0.0624, lr_0 = 6.3280e-04
Validation auc = 0.660094
Epoch 275
Validation auc = 0.660094
Epoch 276
Validation auc = 0.659624
Epoch 277
Validation auc = 0.661033
Epoch 278
Validation auc = 0.660094
Epoch 279
Loss = 7.3400e-03, PNorm = 42.3519, GNorm = 0.0561, lr_0 = 6.2739e-04
Validation auc = 0.653991
Epoch 280
Validation auc = 0.652113
Epoch 281
Validation auc = 0.650704
Epoch 282
Validation auc = 0.651643
Epoch 283
Validation auc = 0.652113
Epoch 284
Loss = 3.1872e-03, PNorm = 42.3844, GNorm = 0.2582, lr_0 = 6.2203e-04
Validation auc = 0.653991
Epoch 285
Validation auc = 0.656808
Epoch 286
Validation auc = 0.656338
Epoch 287
Validation auc = 0.656338
Epoch 288
Validation auc = 0.654460
Epoch 289
Loss = 1.1500e-03, PNorm = 42.4151, GNorm = 0.0209, lr_0 = 6.1671e-04
Validation auc = 0.654930
Epoch 290
Validation auc = 0.655869
Epoch 291
Validation auc = 0.656808
Epoch 292
Validation auc = 0.662441
Epoch 293
Validation auc = 0.669014
Epoch 294
Loss = 6.3063e-03, PNorm = 42.4396, GNorm = 0.4722, lr_0 = 6.1144e-04
Validation auc = 0.669484
Epoch 295
Validation auc = 0.665728
Epoch 296
Validation auc = 0.661033
Epoch 297
Validation auc = 0.660094
Epoch 298
Validation auc = 0.659624
Epoch 299
Loss = 1.1127e-03, PNorm = 42.4693, GNorm = 0.0331, lr_0 = 6.0621e-04
Validation auc = 0.658685
Model 0 best validation auc = 0.746479 on epoch 1
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test auc = 0.721252
Ensemble test auc = 0.721252
Fold 6
Splitting data with seed 6
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.05, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=300, bias=False)
        (W_h): Linear(in_features=300, out_features=300, bias=False)
        (W_o): Linear(in_features=433, out_features=300, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.05, inplace=False)
    (1): Linear(in_features=2200, out_features=300, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.05, inplace=False)
    (4): Linear(in_features=300, out_features=1, bias=True)
  )
)
Number of parameters = 924,901
Moving model to cuda
Epoch 0
Validation auc = 0.661033
Epoch 1
Validation auc = 0.695305
Epoch 2
Validation auc = 0.673239
Epoch 3
Validation auc = 0.735211
Epoch 4
Loss = 8.0194e-01, PNorm = 37.1947, GNorm = 5.0776, lr_0 = 6.5000e-04
Validation auc = 0.698592
Epoch 5
Validation auc = 0.705634
Epoch 6
Validation auc = 0.713615
Epoch 7
Validation auc = 0.665258
Epoch 8
Validation auc = 0.656338
Epoch 9
Loss = 5.1881e-01, PNorm = 37.2535, GNorm = 11.7776, lr_0 = 9.9743e-04
Validation auc = 0.623944
Epoch 10
Validation auc = 0.612676
Epoch 11
Validation auc = 0.613615
Epoch 12
Validation auc = 0.622066
Epoch 13
Validation auc = 0.611268
Epoch 14
Loss = 5.6122e-01, PNorm = 37.3552, GNorm = 6.5765, lr_0 = 9.8890e-04
Validation auc = 0.629108
Epoch 15
Validation auc = 0.572770
Epoch 16
Validation auc = 0.624883
Epoch 17
Validation auc = 0.634742
Epoch 18
Validation auc = 0.634272
Epoch 19
Loss = 5.6646e-01, PNorm = 37.4829, GNorm = 5.9396, lr_0 = 9.8045e-04
Validation auc = 0.640376
Epoch 20
Validation auc = 0.634272
Epoch 21
Validation auc = 0.601408
Epoch 22
Validation auc = 0.634742
Epoch 23
Validation auc = 0.640845
Epoch 24
Loss = 3.6435e-01, PNorm = 37.6202, GNorm = 2.4310, lr_0 = 9.7207e-04
Validation auc = 0.631455
Epoch 25
Validation auc = 0.639906
Epoch 26
Validation auc = 0.638028
Epoch 27
Validation auc = 0.630986
Epoch 28
Validation auc = 0.626291
Epoch 29
Loss = 3.1378e-01, PNorm = 37.7729, GNorm = 2.7136, lr_0 = 9.6376e-04
Validation auc = 0.615493
Epoch 30
Validation auc = 0.615023
Epoch 31
Validation auc = 0.630986
Epoch 32
Validation auc = 0.636150
Epoch 33
Validation auc = 0.627700
Epoch 34
Loss = 3.1702e-01, PNorm = 37.9535, GNorm = 2.5162, lr_0 = 9.5552e-04
Validation auc = 0.633803
Epoch 35
Validation auc = 0.649765
Epoch 36
Validation auc = 0.638967
Epoch 37
Validation auc = 0.632394
Epoch 38
Validation auc = 0.629577
Epoch 39
Loss = 2.9149e-01, PNorm = 38.1356, GNorm = 2.5565, lr_0 = 9.4735e-04
Validation auc = 0.628638
Epoch 40
Validation auc = 0.633333
Epoch 41
Validation auc = 0.624883
Epoch 42
Validation auc = 0.616901
Epoch 43
Validation auc = 0.613146
Epoch 44
Loss = 1.9732e-01, PNorm = 38.3263, GNorm = 1.6083, lr_0 = 9.3925e-04
Validation auc = 0.606573
Epoch 45
Validation auc = 0.625352
Epoch 46
Validation auc = 0.638028
Epoch 47
Validation auc = 0.659624
Epoch 48
Validation auc = 0.646948
Epoch 49
Loss = 2.6721e-01, PNorm = 38.5195, GNorm = 3.6706, lr_0 = 9.3122e-04
Validation auc = 0.631455
Epoch 50
Validation auc = 0.617840
Epoch 51
Validation auc = 0.617840
Epoch 52
Validation auc = 0.607512
Epoch 53
Validation auc = 0.582629
Epoch 54
Loss = 1.3841e-01, PNorm = 38.7073, GNorm = 1.8860, lr_0 = 9.2326e-04
Validation auc = 0.607981
Epoch 55
Validation auc = 0.614085
Epoch 56
Validation auc = 0.612207
Epoch 57
Validation auc = 0.623944
Epoch 58
Validation auc = 0.630047
Epoch 59
Loss = 1.4174e-01, PNorm = 38.9026, GNorm = 0.8791, lr_0 = 9.1537e-04
Validation auc = 0.640376
Epoch 60
Validation auc = 0.641784
Epoch 61
Validation auc = 0.640845
Epoch 62
Validation auc = 0.630516
Epoch 63
Validation auc = 0.628169
Epoch 64
Loss = 1.8586e-01, PNorm = 39.0855, GNorm = 2.1619, lr_0 = 9.0754e-04
Validation auc = 0.633333
Epoch 65
Validation auc = 0.636620
Epoch 66
Validation auc = 0.643662
Epoch 67
Validation auc = 0.634742
Epoch 68
Validation auc = 0.655399
Epoch 69
Loss = 9.4533e-02, PNorm = 39.2611, GNorm = 5.0627, lr_0 = 8.9978e-04
Validation auc = 0.649296
Epoch 70
Validation auc = 0.653521
Epoch 71
Validation auc = 0.653052
Epoch 72
Validation auc = 0.640376
Epoch 73
Validation auc = 0.635211
Epoch 74
Loss = 1.1189e-01, PNorm = 39.4233, GNorm = 0.7635, lr_0 = 8.9209e-04
Validation auc = 0.637559
Epoch 75
Validation auc = 0.640845
Epoch 76
Validation auc = 0.640376
Epoch 77
Validation auc = 0.641784
Epoch 78
Validation auc = 0.638028
Epoch 79
Loss = 6.3302e-02, PNorm = 39.5881, GNorm = 1.9511, lr_0 = 8.8447e-04
Validation auc = 0.644601
Epoch 80
Validation auc = 0.641784
Epoch 81
Validation auc = 0.640376
Epoch 82
Validation auc = 0.633333
Epoch 83
Validation auc = 0.649296
Epoch 84
Loss = 6.8388e-02, PNorm = 39.7370, GNorm = 0.6047, lr_0 = 8.7691e-04
Validation auc = 0.646479
Epoch 85
Validation auc = 0.643192
Epoch 86
Validation auc = 0.642723
Epoch 87
Validation auc = 0.637559
Epoch 88
Validation auc = 0.632864
Epoch 89
Loss = 9.9728e-02, PNorm = 39.8959, GNorm = 0.9269, lr_0 = 8.6941e-04
Validation auc = 0.623474
Epoch 90
Validation auc = 0.660563
Epoch 91
Validation auc = 0.655869
Epoch 92
Validation auc = 0.645540
Epoch 93
Validation auc = 0.641315
Epoch 94
Loss = 1.2796e-01, PNorm = 40.0592, GNorm = 3.8598, lr_0 = 8.6198e-04
Validation auc = 0.636620
Epoch 95
Validation auc = 0.651174
Epoch 96
Validation auc = 0.647418
Epoch 97
Validation auc = 0.644131
Epoch 98
Validation auc = 0.638028
Epoch 99
Loss = 9.7789e-02, PNorm = 40.2212, GNorm = 2.4271, lr_0 = 8.5461e-04
Validation auc = 0.637089
Epoch 100
Validation auc = 0.629108
Epoch 101
Validation auc = 0.636620
Epoch 102
Validation auc = 0.628169
Epoch 103
Validation auc = 0.629108
Epoch 104
Loss = 3.2043e-02, PNorm = 40.3941, GNorm = 0.3432, lr_0 = 8.4730e-04
Validation auc = 0.646948
Epoch 105
Validation auc = 0.656338
Epoch 106
Validation auc = 0.655399
Epoch 107
Validation auc = 0.661502
Epoch 108
Validation auc = 0.652113
Epoch 109
Loss = 9.8755e-02, PNorm = 40.5276, GNorm = 2.2482, lr_0 = 8.4006e-04
Validation auc = 0.654460
Epoch 110
Validation auc = 0.664789
Epoch 111
Validation auc = 0.634272
Epoch 112
Validation auc = 0.644131
Epoch 113
Validation auc = 0.647418
Epoch 114
Loss = 4.1713e-02, PNorm = 40.6429, GNorm = 1.9377, lr_0 = 8.3288e-04
Validation auc = 0.641315
Epoch 115
Validation auc = 0.641784
Epoch 116
Validation auc = 0.634272
Epoch 117
Validation auc = 0.643192
Epoch 118
Validation auc = 0.650235
Epoch 119
Loss = 3.3256e-02, PNorm = 40.7641, GNorm = 1.3519, lr_0 = 8.2576e-04
Validation auc = 0.650235
Epoch 120
Validation auc = 0.641784
Epoch 121
Validation auc = 0.639906
Epoch 122
Validation auc = 0.642723
Epoch 123
Validation auc = 0.649296
Epoch 124
Loss = 4.0012e-02, PNorm = 40.8770, GNorm = 1.0714, lr_0 = 8.1870e-04
Validation auc = 0.645540
Epoch 125
Validation auc = 0.639906
Epoch 126
Validation auc = 0.649296
Epoch 127
Validation auc = 0.653521
Epoch 128
Validation auc = 0.647418
Epoch 129
Loss = 2.6123e-02, PNorm = 40.9719, GNorm = 0.4152, lr_0 = 8.1170e-04
Validation auc = 0.645070
Epoch 130
Validation auc = 0.643192
Epoch 131
Validation auc = 0.654460
Epoch 132
Validation auc = 0.655399
Epoch 133
Validation auc = 0.649765
Epoch 134
Loss = 1.1070e-02, PNorm = 41.0540, GNorm = 0.1983, lr_0 = 8.0476e-04
Validation auc = 0.657277
Epoch 135
Validation auc = 0.658685
Epoch 136
Validation auc = 0.656808
Epoch 137
Validation auc = 0.654460
Epoch 138
Validation auc = 0.651643
Epoch 139
Loss = 3.7919e-02, PNorm = 41.1166, GNorm = 0.3286, lr_0 = 7.9788e-04
Validation auc = 0.661502
Epoch 140
Validation auc = 0.662441
Epoch 141
Validation auc = 0.655869
Epoch 142
Validation auc = 0.652113
Epoch 143
Validation auc = 0.655869
Epoch 144
Loss = 1.1124e-02, PNorm = 41.1795, GNorm = 0.6953, lr_0 = 7.9106e-04
Validation auc = 0.660563
Epoch 145
Validation auc = 0.655869
Epoch 146
Validation auc = 0.653052
Epoch 147
Validation auc = 0.653991
Epoch 148
Validation auc = 0.652113
Epoch 149
Loss = 7.7169e-03, PNorm = 41.2426, GNorm = 0.3588, lr_0 = 7.8430e-04
Validation auc = 0.650704
Epoch 150
Validation auc = 0.653991
Epoch 151
Validation auc = 0.658216
Epoch 152
Validation auc = 0.650704
Epoch 153
Validation auc = 0.641315
Epoch 154
Loss = 4.8845e-02, PNorm = 41.3165, GNorm = 0.6769, lr_0 = 7.7759e-04
Validation auc = 0.653521
Epoch 155
Validation auc = 0.651643
Epoch 156
Validation auc = 0.644601
Epoch 157
Validation auc = 0.655869
Epoch 158
Validation auc = 0.669014
Epoch 159
Loss = 4.4775e-02, PNorm = 41.4143, GNorm = 2.5882, lr_0 = 7.7095e-04
Validation auc = 0.666667
Epoch 160
Validation auc = 0.657277
Epoch 161
Validation auc = 0.655399
Epoch 162
Validation auc = 0.655399
Epoch 163
Validation auc = 0.655869
Epoch 164
Loss = 1.6360e-02, PNorm = 41.5006, GNorm = 0.3146, lr_0 = 7.6436e-04
Validation auc = 0.657746
Epoch 165
Validation auc = 0.658216
Epoch 166
Validation auc = 0.658685
Epoch 167
Validation auc = 0.661502
Epoch 168
Validation auc = 0.656808
Epoch 169
Loss = 2.2088e-02, PNorm = 41.5751, GNorm = 0.7003, lr_0 = 7.5782e-04
Validation auc = 0.654460
Epoch 170
Validation auc = 0.651174
Epoch 171
Validation auc = 0.649296
Epoch 172
Validation auc = 0.655399
Epoch 173
Validation auc = 0.654930
Epoch 174
Loss = 1.7158e-02, PNorm = 41.6438, GNorm = 1.0209, lr_0 = 7.5134e-04
Validation auc = 0.653521
Epoch 175
Validation auc = 0.652582
Epoch 176
Validation auc = 0.651174
Epoch 177
Validation auc = 0.649765
Epoch 178
Validation auc = 0.655399
Epoch 179
Loss = 6.1914e-03, PNorm = 41.6997, GNorm = 0.1723, lr_0 = 7.4492e-04
Validation auc = 0.661033
Epoch 180
Validation auc = 0.662441
Epoch 181
Validation auc = 0.661972
Epoch 182
Validation auc = 0.657277
Epoch 183
Validation auc = 0.656338
Epoch 184
Loss = 7.5952e-03, PNorm = 41.7429, GNorm = 0.2383, lr_0 = 7.3855e-04
Validation auc = 0.656338
Epoch 185
Validation auc = 0.667136
Epoch 186
Validation auc = 0.665258
Epoch 187
Validation auc = 0.657277
Epoch 188
Validation auc = 0.649296
Epoch 189
Loss = 2.3387e-03, PNorm = 41.7864, GNorm = 0.0315, lr_0 = 7.3224e-04
Validation auc = 0.644131
Epoch 190
Validation auc = 0.653052
Epoch 191
Validation auc = 0.658685
Epoch 192
Validation auc = 0.660563
Epoch 193
Validation auc = 0.663380
Epoch 194
Loss = 4.7514e-03, PNorm = 41.8299, GNorm = 0.1550, lr_0 = 7.2598e-04
Validation auc = 0.658216
Epoch 195
Validation auc = 0.657277
Epoch 196
Validation auc = 0.659155
Epoch 197
Validation auc = 0.658216
Epoch 198
Validation auc = 0.659624
Epoch 199
Loss = 5.6115e-03, PNorm = 41.8722, GNorm = 0.1665, lr_0 = 7.1977e-04
Validation auc = 0.659155
Epoch 200
Validation auc = 0.659155
Epoch 201
Validation auc = 0.660094
Epoch 202
Validation auc = 0.661033
Epoch 203
Validation auc = 0.661972
Epoch 204
Loss = 4.4897e-03, PNorm = 41.9084, GNorm = 0.1102, lr_0 = 7.1362e-04
Validation auc = 0.661033
Epoch 205
Validation auc = 0.663380
Epoch 206
Validation auc = 0.663380
Epoch 207
Validation auc = 0.663380
Epoch 208
Validation auc = 0.668545
Epoch 209
Loss = 4.1450e-03, PNorm = 41.9373, GNorm = 0.2561, lr_0 = 7.0752e-04
Validation auc = 0.671362
Epoch 210
Validation auc = 0.666667
Epoch 211
Validation auc = 0.664319
Epoch 212
Validation auc = 0.664319
Epoch 213
Validation auc = 0.670423
Epoch 214
Loss = 1.8252e-03, PNorm = 41.9660, GNorm = 0.0601, lr_0 = 7.0147e-04
Validation auc = 0.673709
Epoch 215
Validation auc = 0.673709
Epoch 216
Validation auc = 0.673709
Epoch 217
Validation auc = 0.670892
Epoch 218
Validation auc = 0.671362
Epoch 219
Loss = 3.4471e-03, PNorm = 41.9926, GNorm = 0.2114, lr_0 = 6.9548e-04
Validation auc = 0.670892
Epoch 220
Validation auc = 0.672300
Epoch 221
Validation auc = 0.673239
Epoch 222
Validation auc = 0.672770
Epoch 223
Validation auc = 0.673709
Epoch 224
Loss = 3.4648e-03, PNorm = 42.0143, GNorm = 0.1193, lr_0 = 6.8953e-04
Validation auc = 0.674178
Epoch 225
Validation auc = 0.670892
Epoch 226
Validation auc = 0.667606
Epoch 227
Validation auc = 0.668075
Epoch 228
Validation auc = 0.673239
Epoch 229
Loss = 1.3953e-03, PNorm = 42.0352, GNorm = 0.0342, lr_0 = 6.8364e-04
Validation auc = 0.674648
Epoch 230
Validation auc = 0.673239
Epoch 231
Validation auc = 0.671831
Epoch 232
Validation auc = 0.671362
Epoch 233
Validation auc = 0.671831
Epoch 234
Loss = 2.4366e-03, PNorm = 42.0542, GNorm = 0.0494, lr_0 = 6.7779e-04
Validation auc = 0.672300
Epoch 235
Validation auc = 0.669484
Epoch 236
Validation auc = 0.669484
Epoch 237
Validation auc = 0.671362
Epoch 238
Validation auc = 0.664789
Epoch 239
Loss = 1.9152e-03, PNorm = 42.0738, GNorm = 0.1049, lr_0 = 6.7200e-04
Validation auc = 0.661502
Epoch 240
Validation auc = 0.669953
Epoch 241
Validation auc = 0.669953
Epoch 242
Validation auc = 0.666667
Epoch 243
Validation auc = 0.658685
Epoch 244
Loss = 1.5482e-03, PNorm = 42.1075, GNorm = 0.0247, lr_0 = 6.6625e-04
Validation auc = 0.656338
Epoch 245
Validation auc = 0.656338
Epoch 246
Validation auc = 0.656338
Epoch 247
Validation auc = 0.658216
Epoch 248
Validation auc = 0.661972
Epoch 249
Loss = 6.0678e-03, PNorm = 42.1343, GNorm = 0.5472, lr_0 = 6.6056e-04
Validation auc = 0.670423
Epoch 250
Validation auc = 0.671362
Epoch 251
Validation auc = 0.669014
Epoch 252
Validation auc = 0.668545
Epoch 253
Validation auc = 0.666667
Epoch 254
Loss = 1.0804e-03, PNorm = 42.1608, GNorm = 0.0440, lr_0 = 6.5491e-04
Validation auc = 0.665728
Epoch 255
Validation auc = 0.668545
Epoch 256
Validation auc = 0.670892
Epoch 257
Validation auc = 0.672300
Epoch 258
Validation auc = 0.671831
Epoch 259
Loss = 2.3057e-03, PNorm = 42.1804, GNorm = 0.0463, lr_0 = 6.4931e-04
Validation auc = 0.670423
Epoch 260
Validation auc = 0.667136
Epoch 261
Validation auc = 0.666667
Epoch 262
Validation auc = 0.664319
Epoch 263
Validation auc = 0.663850
Epoch 264
Loss = 1.1653e-03, PNorm = 42.1965, GNorm = 0.0597, lr_0 = 6.4376e-04
Validation auc = 0.663380
Epoch 265
Validation auc = 0.661972
Epoch 266
Validation auc = 0.662441
Epoch 267
Validation auc = 0.663850
Epoch 268
Validation auc = 0.665728
Epoch 269
Loss = 2.4260e-03, PNorm = 42.2106, GNorm = 0.0474, lr_0 = 6.3826e-04
Validation auc = 0.665258
Epoch 270
Validation auc = 0.667136
Epoch 271
Validation auc = 0.669014
Epoch 272
Validation auc = 0.669953
Epoch 273
Validation auc = 0.670892
Epoch 274
Loss = 7.0738e-03, PNorm = 42.2235, GNorm = 0.5968, lr_0 = 6.3280e-04
Validation auc = 0.673239
Epoch 275
Validation auc = 0.676526
Epoch 276
Validation auc = 0.675587
Epoch 277
Validation auc = 0.673709
Epoch 278
Validation auc = 0.671362
Epoch 279
Loss = 1.3455e-03, PNorm = 42.2452, GNorm = 0.0269, lr_0 = 6.2739e-04
Validation auc = 0.669953
Epoch 280
Validation auc = 0.669014
Epoch 281
Validation auc = 0.667606
Epoch 282
Validation auc = 0.668545
Epoch 283
Validation auc = 0.668075
Epoch 284
Loss = 1.4321e-03, PNorm = 42.2636, GNorm = 0.0611, lr_0 = 6.2203e-04
Validation auc = 0.669484
Epoch 285
Validation auc = 0.669953
Epoch 286
Validation auc = 0.669484
Epoch 287
Validation auc = 0.669014
Epoch 288
Validation auc = 0.667136
Epoch 289
Loss = 9.7077e-04, PNorm = 42.2816, GNorm = 0.0189, lr_0 = 6.1671e-04
Validation auc = 0.667606
Epoch 290
Validation auc = 0.667136
Epoch 291
Validation auc = 0.666667
Epoch 292
Validation auc = 0.667136
Epoch 293
Validation auc = 0.668545
Epoch 294
Loss = 1.9183e-03, PNorm = 42.2948, GNorm = 0.1609, lr_0 = 6.1144e-04
Validation auc = 0.669014
Epoch 295
Validation auc = 0.674178
Epoch 296
Validation auc = 0.673239
Epoch 297
Validation auc = 0.674648
Epoch 298
Validation auc = 0.673239
Epoch 299
Loss = 1.2832e-03, PNorm = 42.3108, GNorm = 0.0320, lr_0 = 6.0621e-04
Validation auc = 0.672300
Model 0 best validation auc = 0.735211 on epoch 3
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test auc = 0.754365
Ensemble test auc = 0.754365
Fold 7
Splitting data with seed 7
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.05, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=300, bias=False)
        (W_h): Linear(in_features=300, out_features=300, bias=False)
        (W_o): Linear(in_features=433, out_features=300, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.05, inplace=False)
    (1): Linear(in_features=2200, out_features=300, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.05, inplace=False)
    (4): Linear(in_features=300, out_features=1, bias=True)
  )
)
Number of parameters = 924,901
Moving model to cuda
Epoch 0
Validation auc = 0.661972
Epoch 1
Validation auc = 0.669953
Epoch 2
Validation auc = 0.723944
Epoch 3
Validation auc = 0.686385
Epoch 4
Loss = 8.3363e-01, PNorm = 37.1913, GNorm = 2.9497, lr_0 = 6.5000e-04
Validation auc = 0.698122
Epoch 5
Validation auc = 0.708920
Epoch 6
Validation auc = 0.667606
Epoch 7
Validation auc = 0.657746
Epoch 8
Validation auc = 0.568075
Epoch 9
Loss = 7.8720e-01, PNorm = 37.2477, GNorm = 4.5226, lr_0 = 9.9743e-04
Validation auc = 0.635211
Epoch 10
Validation auc = 0.633333
Epoch 11
Validation auc = 0.607042
Epoch 12
Validation auc = 0.608451
Epoch 13
Validation auc = 0.636150
Epoch 14
Loss = 6.3796e-01, PNorm = 37.3473, GNorm = 7.9591, lr_0 = 9.8890e-04
Validation auc = 0.637559
Epoch 15
Validation auc = 0.641784
Epoch 16
Validation auc = 0.636620
Epoch 17
Validation auc = 0.647887
Epoch 18
Validation auc = 0.647887
Epoch 19
Loss = 4.4917e-01, PNorm = 37.4590, GNorm = 3.6335, lr_0 = 9.8045e-04
Validation auc = 0.640376
Epoch 20
Validation auc = 0.628169
Epoch 21
Validation auc = 0.638967
Epoch 22
Validation auc = 0.640376
Epoch 23
Validation auc = 0.634742
Epoch 24
Loss = 3.4783e-01, PNorm = 37.5915, GNorm = 5.5050, lr_0 = 9.7207e-04
Validation auc = 0.647887
Epoch 25
Validation auc = 0.641315
Epoch 26
Validation auc = 0.639906
Epoch 27
Validation auc = 0.647887
Epoch 28
Validation auc = 0.639906
Epoch 29
Loss = 3.2675e-01, PNorm = 37.7170, GNorm = 5.3112, lr_0 = 9.6376e-04
Validation auc = 0.636150
Epoch 30
Validation auc = 0.634742
Epoch 31
Validation auc = 0.609390
Epoch 32
Validation auc = 0.607981
Epoch 33
Validation auc = 0.629108
Epoch 34
Loss = 3.5726e-01, PNorm = 37.8657, GNorm = 4.5389, lr_0 = 9.5552e-04
Validation auc = 0.631455
Epoch 35
Validation auc = 0.630986
Epoch 36
Validation auc = 0.625822
Epoch 37
Validation auc = 0.632864
Epoch 38
Validation auc = 0.637559
Epoch 39
Loss = 3.0521e-01, PNorm = 38.0432, GNorm = 2.2899, lr_0 = 9.4735e-04
Validation auc = 0.630986
Epoch 40
Validation auc = 0.631455
Epoch 41
Validation auc = 0.630047
Epoch 42
Validation auc = 0.627700
Epoch 43
Validation auc = 0.619249
Epoch 44
Loss = 2.3402e-01, PNorm = 38.2383, GNorm = 4.4356, lr_0 = 9.3925e-04
Validation auc = 0.631925
Epoch 45
Validation auc = 0.623474
Epoch 46
Validation auc = 0.613146
Epoch 47
Validation auc = 0.624413
Epoch 48
Validation auc = 0.621596
Epoch 49
Loss = 2.2549e-01, PNorm = 38.4475, GNorm = 3.7146, lr_0 = 9.3122e-04
Validation auc = 0.625822
Epoch 50
Validation auc = 0.636620
Epoch 51
Validation auc = 0.622535
Epoch 52
Validation auc = 0.624413
Epoch 53
Validation auc = 0.622066
Epoch 54
Loss = 2.0049e-01, PNorm = 38.6606, GNorm = 1.0423, lr_0 = 9.2326e-04
Validation auc = 0.615493
Epoch 55
Validation auc = 0.618310
Epoch 56
Validation auc = 0.624413
Epoch 57
Validation auc = 0.630986
Epoch 58
Validation auc = 0.634272
Epoch 59
Loss = 1.9134e-01, PNorm = 38.8911, GNorm = 2.3556, lr_0 = 9.1537e-04
Validation auc = 0.634742
Epoch 60
Validation auc = 0.632864
Epoch 61
Validation auc = 0.638498
Epoch 62
Validation auc = 0.637089
Epoch 63
Validation auc = 0.635211
Epoch 64
Loss = 1.1715e-01, PNorm = 39.0955, GNorm = 2.1134, lr_0 = 9.0754e-04
Validation auc = 0.637559
Epoch 65
Validation auc = 0.638028
Epoch 66
Validation auc = 0.623005
Epoch 67
Validation auc = 0.638028
Epoch 68
Validation auc = 0.633803
Epoch 69
Loss = 2.0117e-01, PNorm = 39.3021, GNorm = 6.0660, lr_0 = 8.9978e-04
Validation auc = 0.637559
Epoch 70
Validation auc = 0.638967
Epoch 71
Validation auc = 0.641315
Epoch 72
Validation auc = 0.636150
Epoch 73
Validation auc = 0.630986
Epoch 74
Loss = 1.7287e-01, PNorm = 39.5152, GNorm = 2.8490, lr_0 = 8.9209e-04
Validation auc = 0.619249
Epoch 75
Validation auc = 0.614554
Epoch 76
Validation auc = 0.628638
Epoch 77
Validation auc = 0.637559
Epoch 78
Validation auc = 0.639906
Epoch 79
Loss = 9.6913e-02, PNorm = 39.7090, GNorm = 1.7681, lr_0 = 8.8447e-04
Validation auc = 0.648826
Epoch 80
Validation auc = 0.659624
Epoch 81
Validation auc = 0.654930
Epoch 82
Validation auc = 0.649296
Epoch 83
Validation auc = 0.640845
Epoch 84
Loss = 1.3832e-01, PNorm = 39.8995, GNorm = 3.6238, lr_0 = 8.7691e-04
Validation auc = 0.644131
Epoch 85
Validation auc = 0.641315
Epoch 86
Validation auc = 0.627230
Epoch 87
Validation auc = 0.628169
Epoch 88
Validation auc = 0.628169
Epoch 89
Loss = 9.1431e-02, PNorm = 40.0642, GNorm = 0.4059, lr_0 = 8.6941e-04
Validation auc = 0.640376
Epoch 90
Validation auc = 0.631925
Epoch 91
Validation auc = 0.630047
Epoch 92
Validation auc = 0.633803
Epoch 93
Validation auc = 0.637089
Epoch 94
Loss = 5.1588e-02, PNorm = 40.2053, GNorm = 0.9036, lr_0 = 8.6198e-04
Validation auc = 0.642254
Epoch 95
Validation auc = 0.650235
Epoch 96
Validation auc = 0.652582
Epoch 97
Validation auc = 0.656338
Epoch 98
Validation auc = 0.653521
Epoch 99
Loss = 5.8267e-02, PNorm = 40.3412, GNorm = 0.8016, lr_0 = 8.5461e-04
Validation auc = 0.647418
Epoch 100
Validation auc = 0.642723
Epoch 101
Validation auc = 0.638028
Epoch 102
Validation auc = 0.642254
Epoch 103
Validation auc = 0.638028
Epoch 104
Loss = 3.2287e-02, PNorm = 40.4666, GNorm = 0.3359, lr_0 = 8.4730e-04
Validation auc = 0.631925
Epoch 105
Validation auc = 0.639437
Epoch 106
Validation auc = 0.642254
Epoch 107
Validation auc = 0.647418
Epoch 108
Validation auc = 0.656808
Epoch 109
Loss = 2.1587e-02, PNorm = 40.5870, GNorm = 1.1249, lr_0 = 8.4006e-04
Validation auc = 0.651643
Epoch 110
Validation auc = 0.647887
Epoch 111
Validation auc = 0.647418
Epoch 112
Validation auc = 0.653521
Epoch 113
Validation auc = 0.640376
Epoch 114
Loss = 7.6528e-02, PNorm = 40.6840, GNorm = 2.3058, lr_0 = 8.3288e-04
Validation auc = 0.640376
Epoch 115
Validation auc = 0.643192
Epoch 116
Validation auc = 0.639437
Epoch 117
Validation auc = 0.635681
Epoch 118
Validation auc = 0.638498
Epoch 119
Loss = 2.9024e-02, PNorm = 40.7922, GNorm = 1.0334, lr_0 = 8.2576e-04
Validation auc = 0.649765
Epoch 120
Validation auc = 0.644601
Epoch 121
Validation auc = 0.650235
Epoch 122
Validation auc = 0.650235
Epoch 123
Validation auc = 0.645070
Epoch 124
Loss = 1.6392e-02, PNorm = 40.9107, GNorm = 0.3665, lr_0 = 8.1870e-04
Validation auc = 0.643192
Epoch 125
Validation auc = 0.646479
Epoch 126
Validation auc = 0.653052
Epoch 127
Validation auc = 0.654930
Epoch 128
Validation auc = 0.651643
Epoch 129
Loss = 6.4358e-02, PNorm = 41.0466, GNorm = 1.5404, lr_0 = 8.1170e-04
Validation auc = 0.649765
Epoch 130
Validation auc = 0.652582
Epoch 131
Validation auc = 0.637559
Epoch 132
Validation auc = 0.633333
Epoch 133
Validation auc = 0.625352
Epoch 134
Loss = 4.8091e-02, PNorm = 41.1817, GNorm = 0.6820, lr_0 = 8.0476e-04
Validation auc = 0.621596
Epoch 135
Validation auc = 0.618779
Epoch 136
Validation auc = 0.622535
Epoch 137
Validation auc = 0.626291
Epoch 138
Validation auc = 0.628169
Epoch 139
Loss = 3.3112e-02, PNorm = 41.2946, GNorm = 0.6154, lr_0 = 7.9788e-04
Validation auc = 0.638028
Epoch 140
Validation auc = 0.646009
Epoch 141
Validation auc = 0.653991
Epoch 142
Validation auc = 0.657277
Epoch 143
Validation auc = 0.666667
Epoch 144
Loss = 1.5378e-02, PNorm = 41.3895, GNorm = 0.3290, lr_0 = 7.9106e-04
Validation auc = 0.661972
Epoch 145
Validation auc = 0.658216
Epoch 146
Validation auc = 0.652113
Epoch 147
Validation auc = 0.652582
Epoch 148
Validation auc = 0.651174
Epoch 149
Loss = 3.0114e-02, PNorm = 41.4684, GNorm = 0.1731, lr_0 = 7.8430e-04
Validation auc = 0.647887
Epoch 150
Validation auc = 0.643192
Epoch 151
Validation auc = 0.637089
Epoch 152
Validation auc = 0.625822
Epoch 153
Validation auc = 0.624883
Epoch 154
Loss = 2.7056e-02, PNorm = 41.5636, GNorm = 0.6174, lr_0 = 7.7759e-04
Validation auc = 0.638028
Epoch 155
Validation auc = 0.647887
Epoch 156
Validation auc = 0.645540
Epoch 157
Validation auc = 0.635211
Epoch 158
Validation auc = 0.636620
Epoch 159
Loss = 1.5732e-02, PNorm = 41.6579, GNorm = 0.4109, lr_0 = 7.7095e-04
Validation auc = 0.639437
Epoch 160
Validation auc = 0.645540
Epoch 161
Validation auc = 0.654460
Epoch 162
Validation auc = 0.655399
Epoch 163
Validation auc = 0.655869
Epoch 164
Loss = 5.3896e-03, PNorm = 41.7540, GNorm = 0.1684, lr_0 = 7.6436e-04
Validation auc = 0.656338
Epoch 165
Validation auc = 0.661502
Epoch 166
Validation auc = 0.670892
Epoch 167
Validation auc = 0.658216
Epoch 168
Validation auc = 0.648826
Epoch 169
Loss = 1.9267e-02, PNorm = 41.8229, GNorm = 0.3835, lr_0 = 7.5782e-04
Validation auc = 0.643662
Epoch 170
Validation auc = 0.649765
Epoch 171
Validation auc = 0.649765
Epoch 172
Validation auc = 0.641784
Epoch 173
Validation auc = 0.638028
Epoch 174
Loss = 1.0325e-02, PNorm = 41.8956, GNorm = 0.2859, lr_0 = 7.5134e-04
Validation auc = 0.640845
Epoch 175
Validation auc = 0.638967
Epoch 176
Validation auc = 0.644601
Epoch 177
Validation auc = 0.641784
Epoch 178
Validation auc = 0.637559
Epoch 179
Loss = 1.0551e-02, PNorm = 41.9740, GNorm = 0.5136, lr_0 = 7.4492e-04
Validation auc = 0.636150
Epoch 180
Validation auc = 0.641784
Epoch 181
Validation auc = 0.645070
Epoch 182
Validation auc = 0.643192
Epoch 183
Validation auc = 0.645540
Epoch 184
Loss = 8.5664e-03, PNorm = 42.0590, GNorm = 0.4396, lr_0 = 7.3855e-04
Validation auc = 0.641784
Epoch 185
Validation auc = 0.644131
Epoch 186
Validation auc = 0.650235
Epoch 187
Validation auc = 0.657746
Epoch 188
Validation auc = 0.655399
Epoch 189
Loss = 7.5538e-03, PNorm = 42.1324, GNorm = 0.1033, lr_0 = 7.3224e-04
Validation auc = 0.652582
Epoch 190
Validation auc = 0.645070
Epoch 191
Validation auc = 0.641784
Epoch 192
Validation auc = 0.642254
Epoch 193
Validation auc = 0.647887
Epoch 194
Loss = 2.5830e-02, PNorm = 42.1851, GNorm = 2.6733, lr_0 = 7.2598e-04
Validation auc = 0.641784
Epoch 195
Validation auc = 0.637089
Epoch 196
Validation auc = 0.638028
Epoch 197
Validation auc = 0.637089
Epoch 198
Validation auc = 0.634272
Epoch 199
Loss = 9.1421e-03, PNorm = 42.2440, GNorm = 0.3820, lr_0 = 7.1977e-04
Validation auc = 0.639437
Epoch 200
Validation auc = 0.644601
Epoch 201
Validation auc = 0.644131
Epoch 202
Validation auc = 0.643662
Epoch 203
Validation auc = 0.644131
Epoch 204
Loss = 4.3956e-03, PNorm = 42.3136, GNorm = 0.3667, lr_0 = 7.1362e-04
Validation auc = 0.648357
Epoch 205
Validation auc = 0.656338
Epoch 206
Validation auc = 0.658685
Epoch 207
Validation auc = 0.661502
Epoch 208
Validation auc = 0.662441
Epoch 209
Loss = 8.5969e-03, PNorm = 42.3576, GNorm = 0.1520, lr_0 = 7.0752e-04
Validation auc = 0.663380
Epoch 210
Validation auc = 0.661972
Epoch 211
Validation auc = 0.658685
Epoch 212
Validation auc = 0.657746
Epoch 213
Validation auc = 0.654460
Epoch 214
Loss = 1.0094e-02, PNorm = 42.3958, GNorm = 0.3335, lr_0 = 7.0147e-04
Validation auc = 0.653991
Epoch 215
Validation auc = 0.649296
Epoch 216
Validation auc = 0.645070
Epoch 217
Validation auc = 0.646948
Epoch 218
Validation auc = 0.648826
Epoch 219
Loss = 3.5767e-03, PNorm = 42.4390, GNorm = 0.1498, lr_0 = 6.9548e-04
Validation auc = 0.649296
Epoch 220
Validation auc = 0.650235
Epoch 221
Validation auc = 0.652113
Epoch 222
Validation auc = 0.651643
Epoch 223
Validation auc = 0.653991
Epoch 224
Loss = 8.4803e-03, PNorm = 42.4781, GNorm = 0.4454, lr_0 = 6.8953e-04
Validation auc = 0.654460
Epoch 225
Validation auc = 0.654460
Epoch 226
Validation auc = 0.654460
Epoch 227
Validation auc = 0.657277
Epoch 228
Validation auc = 0.658216
Epoch 229
Loss = 3.2318e-03, PNorm = 42.5137, GNorm = 0.0909, lr_0 = 6.8364e-04
Validation auc = 0.660094
Epoch 230
Validation auc = 0.661033
Epoch 231
Validation auc = 0.662441
Epoch 232
Validation auc = 0.660094
Epoch 233
Validation auc = 0.661033
Epoch 234
Loss = 1.7287e-03, PNorm = 42.5404, GNorm = 0.0341, lr_0 = 6.7779e-04
Validation auc = 0.661033
Epoch 235
Validation auc = 0.661033
Epoch 236
Validation auc = 0.661972
Epoch 237
Validation auc = 0.664789
Epoch 238
Validation auc = 0.666197
Epoch 239
Loss = 3.3251e-03, PNorm = 42.5590, GNorm = 0.0439, lr_0 = 6.7200e-04
Validation auc = 0.665728
Epoch 240
Validation auc = 0.665728
Epoch 241
Validation auc = 0.666197
Epoch 242
Validation auc = 0.664789
Epoch 243
Validation auc = 0.665728
Epoch 244
Loss = 2.4894e-03, PNorm = 42.5791, GNorm = 0.0444, lr_0 = 6.6625e-04
Validation auc = 0.662441
Epoch 245
Validation auc = 0.661972
Epoch 246
Validation auc = 0.661502
Epoch 247
Validation auc = 0.660563
Epoch 248
Validation auc = 0.658685
Epoch 249
Loss = 3.5464e-03, PNorm = 42.5998, GNorm = 0.0741, lr_0 = 6.6056e-04
Validation auc = 0.656338
Epoch 250
Validation auc = 0.656808
Epoch 251
Validation auc = 0.655869
Epoch 252
Validation auc = 0.655399
Epoch 253
Validation auc = 0.654460
Epoch 254
Loss = 1.2205e-03, PNorm = 42.6174, GNorm = 0.0304, lr_0 = 6.5491e-04
Validation auc = 0.654930
Epoch 255
Validation auc = 0.657746
Epoch 256
Validation auc = 0.659624
Epoch 257
Validation auc = 0.660094
Epoch 258
Validation auc = 0.661033
Epoch 259
Loss = 1.9649e-03, PNorm = 42.6331, GNorm = 0.0654, lr_0 = 6.4931e-04
Validation auc = 0.661502
Epoch 260
Validation auc = 0.661033
Epoch 261
Validation auc = 0.659155
Epoch 262
Validation auc = 0.658685
Epoch 263
Validation auc = 0.657746
Epoch 264
Loss = 1.8078e-03, PNorm = 42.6496, GNorm = 0.1225, lr_0 = 6.4376e-04
Validation auc = 0.657746
Epoch 265
Validation auc = 0.661502
Epoch 266
Validation auc = 0.661972
Epoch 267
Validation auc = 0.663850
Epoch 268
Validation auc = 0.662911
Epoch 269
Loss = 1.1763e-03, PNorm = 42.6647, GNorm = 0.0205, lr_0 = 6.3826e-04
Validation auc = 0.662911
Epoch 270
Validation auc = 0.661502
Epoch 271
Validation auc = 0.660563
Epoch 272
Validation auc = 0.660094
Epoch 273
Validation auc = 0.661033
Epoch 274
Loss = 3.0141e-03, PNorm = 42.6786, GNorm = 0.0148, lr_0 = 6.3280e-04
Validation auc = 0.661033
Epoch 275
Validation auc = 0.660563
Epoch 276
Validation auc = 0.658685
Epoch 277
Validation auc = 0.656338
Epoch 278
Validation auc = 0.652582
Epoch 279
Loss = 1.2493e-03, PNorm = 42.7007, GNorm = 0.0491, lr_0 = 6.2739e-04
Validation auc = 0.653521
Epoch 280
Validation auc = 0.653521
Epoch 281
Validation auc = 0.654930
Epoch 282
Validation auc = 0.660094
Epoch 283
Validation auc = 0.660563
Epoch 284
Loss = 1.7859e-03, PNorm = 42.7247, GNorm = 0.0514, lr_0 = 6.2203e-04
Validation auc = 0.661502
Epoch 285
Validation auc = 0.663380
Epoch 286
Validation auc = 0.661972
Epoch 287
Validation auc = 0.661502
Epoch 288
Validation auc = 0.663850
Epoch 289
Loss = 1.1144e-03, PNorm = 42.7427, GNorm = 0.0233, lr_0 = 6.1671e-04
Validation auc = 0.665258
Epoch 290
Validation auc = 0.668075
Epoch 291
Validation auc = 0.664789
Epoch 292
Validation auc = 0.660094
Epoch 293
Validation auc = 0.659624
Epoch 294
Loss = 1.3940e-03, PNorm = 42.7621, GNorm = 0.1118, lr_0 = 6.1144e-04
Validation auc = 0.656808
Epoch 295
Validation auc = 0.656338
Epoch 296
Validation auc = 0.655869
Epoch 297
Validation auc = 0.656808
Epoch 298
Validation auc = 0.657277
Epoch 299
Loss = 1.5935e-03, PNorm = 42.7796, GNorm = 0.0967, lr_0 = 6.0621e-04
Validation auc = 0.658216
Model 0 best validation auc = 0.723944 on epoch 2
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test auc = 0.762191
Ensemble test auc = 0.762191
Fold 8
Splitting data with seed 8
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.05, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=300, bias=False)
        (W_h): Linear(in_features=300, out_features=300, bias=False)
        (W_o): Linear(in_features=433, out_features=300, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.05, inplace=False)
    (1): Linear(in_features=2200, out_features=300, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.05, inplace=False)
    (4): Linear(in_features=300, out_features=1, bias=True)
  )
)
Number of parameters = 924,901
Moving model to cuda
Epoch 0
Validation auc = 0.664789
Epoch 1
Validation auc = 0.685915
Epoch 2
Validation auc = 0.710798
Epoch 3
Validation auc = 0.681690
Epoch 4
Loss = 6.3675e-01, PNorm = 37.1929, GNorm = 4.9566, lr_0 = 6.5000e-04
Validation auc = 0.745540
Epoch 5
Validation auc = 0.690610
Epoch 6
Validation auc = 0.681221
Epoch 7
Validation auc = 0.746948
Epoch 8
Validation auc = 0.675587
Epoch 9
Loss = 5.1507e-01, PNorm = 37.2533, GNorm = 5.9054, lr_0 = 9.9743e-04
Validation auc = 0.668545
Epoch 10
Validation auc = 0.674648
Epoch 11
Validation auc = 0.655869
Epoch 12
Validation auc = 0.667136
Epoch 13
Validation auc = 0.666197
Epoch 14
Loss = 5.1635e-01, PNorm = 37.3567, GNorm = 3.7051, lr_0 = 9.8890e-04
Validation auc = 0.649296
Epoch 15
Validation auc = 0.643662
Epoch 16
Validation auc = 0.632394
Epoch 17
Validation auc = 0.623474
Epoch 18
Validation auc = 0.585446
Epoch 19
Loss = 3.9754e-01, PNorm = 37.4908, GNorm = 3.2871, lr_0 = 9.8045e-04
Validation auc = 0.624883
Epoch 20
Validation auc = 0.631925
Epoch 21
Validation auc = 0.615493
Epoch 22
Validation auc = 0.630047
Epoch 23
Validation auc = 0.592019
Epoch 24
Loss = 3.4531e-01, PNorm = 37.6367, GNorm = 4.4094, lr_0 = 9.7207e-04
Validation auc = 0.621127
Epoch 25
Validation auc = 0.642254
Epoch 26
Validation auc = 0.640845
Epoch 27
Validation auc = 0.640845
Epoch 28
Validation auc = 0.645540
Epoch 29
Loss = 3.6217e-01, PNorm = 37.7916, GNorm = 1.6126, lr_0 = 9.6376e-04
Validation auc = 0.629108
Epoch 30
Validation auc = 0.627700
Epoch 31
Validation auc = 0.626291
Epoch 32
Validation auc = 0.597183
Epoch 33
Validation auc = 0.627700
Epoch 34
Loss = 3.7388e-01, PNorm = 37.9631, GNorm = 8.3056, lr_0 = 9.5552e-04
Validation auc = 0.627230
Epoch 35
Validation auc = 0.591080
Epoch 36
Validation auc = 0.632864
Epoch 37
Validation auc = 0.632864
Epoch 38
Validation auc = 0.632394
Epoch 39
Loss = 3.5173e-01, PNorm = 38.1378, GNorm = 9.4294, lr_0 = 9.4735e-04
Validation auc = 0.637559
Epoch 40
Validation auc = 0.641315
Epoch 41
Validation auc = 0.623944
Epoch 42
Validation auc = 0.630047
Epoch 43
Validation auc = 0.638028
Epoch 44
Loss = 2.6176e-01, PNorm = 38.3217, GNorm = 1.6606, lr_0 = 9.3925e-04
Validation auc = 0.640376
Epoch 45
Validation auc = 0.642723
Epoch 46
Validation auc = 0.649296
Epoch 47
Validation auc = 0.639906
Epoch 48
Validation auc = 0.638967
Epoch 49
Loss = 1.9961e-01, PNorm = 38.5202, GNorm = 3.1949, lr_0 = 9.3122e-04
Validation auc = 0.621596
Epoch 50
Validation auc = 0.638028
Epoch 51
Validation auc = 0.633803
Epoch 52
Validation auc = 0.613615
Epoch 53
Validation auc = 0.638028
Epoch 54
Loss = 1.5635e-01, PNorm = 38.7141, GNorm = 2.8052, lr_0 = 9.2326e-04
Validation auc = 0.637559
Epoch 55
Validation auc = 0.634742
Epoch 56
Validation auc = 0.646479
Epoch 57
Validation auc = 0.617371
Epoch 58
Validation auc = 0.632394
Epoch 59
Loss = 1.7813e-01, PNorm = 38.9166, GNorm = 4.3397, lr_0 = 9.1537e-04
Validation auc = 0.633803
Epoch 60
Validation auc = 0.621127
Epoch 61
Validation auc = 0.629577
Epoch 62
Validation auc = 0.626291
Epoch 63
Validation auc = 0.632864
Epoch 64
Loss = 1.2925e-01, PNorm = 39.1150, GNorm = 2.4073, lr_0 = 9.0754e-04
Validation auc = 0.649296
Epoch 65
Validation auc = 0.649765
Epoch 66
Validation auc = 0.645540
Epoch 67
Validation auc = 0.646948
Epoch 68
Validation auc = 0.648826
Epoch 69
Loss = 1.2247e-01, PNorm = 39.3158, GNorm = 0.9163, lr_0 = 8.9978e-04
Validation auc = 0.650235
Epoch 70
Validation auc = 0.653052
Epoch 71
Validation auc = 0.664789
Epoch 72
Validation auc = 0.645540
Epoch 73
Validation auc = 0.636150
Epoch 74
Loss = 1.0728e-01, PNorm = 39.4905, GNorm = 1.5166, lr_0 = 8.9209e-04
Validation auc = 0.627700
Epoch 75
Validation auc = 0.620188
Epoch 76
Validation auc = 0.622535
Epoch 77
Validation auc = 0.626761
Epoch 78
Validation auc = 0.621127
Epoch 79
Loss = 1.0069e-01, PNorm = 39.6742, GNorm = 4.9429, lr_0 = 8.8447e-04
Validation auc = 0.629108
Epoch 80
Validation auc = 0.631925
Epoch 81
Validation auc = 0.627230
Epoch 82
Validation auc = 0.635211
Epoch 83
Validation auc = 0.624413
Epoch 84
Loss = 2.6455e-01, PNorm = 39.8585, GNorm = 11.2792, lr_0 = 8.7691e-04
Validation auc = 0.606573
Epoch 85
Validation auc = 0.641784
Epoch 86
Validation auc = 0.632394
Epoch 87
Validation auc = 0.600469
Epoch 88
Validation auc = 0.594366
Epoch 89
Loss = 1.6971e-01, PNorm = 40.0685, GNorm = 0.9907, lr_0 = 8.6941e-04
Validation auc = 0.613615
Epoch 90
Validation auc = 0.614554
Epoch 91
Validation auc = 0.613615
Epoch 92
Validation auc = 0.614085
Epoch 93
Validation auc = 0.627700
Epoch 94
Loss = 1.2748e-01, PNorm = 40.2891, GNorm = 2.4524, lr_0 = 8.6198e-04
Validation auc = 0.635681
Epoch 95
Validation auc = 0.629108
Epoch 96
Validation auc = 0.603286
Epoch 97
Validation auc = 0.613146
Epoch 98
Validation auc = 0.638498
Epoch 99
Loss = 6.3109e-02, PNorm = 40.4619, GNorm = 2.1449, lr_0 = 8.5461e-04
Validation auc = 0.640845
Epoch 100
Validation auc = 0.645070
Epoch 101
Validation auc = 0.648357
Epoch 102
Validation auc = 0.655399
Epoch 103
Validation auc = 0.658685
Epoch 104
Loss = 3.3482e-02, PNorm = 40.6138, GNorm = 0.2425, lr_0 = 8.4730e-04
Validation auc = 0.655399
Epoch 105
Validation auc = 0.658685
Epoch 106
Validation auc = 0.659155
Epoch 107
Validation auc = 0.657746
Epoch 108
Validation auc = 0.658685
Epoch 109
Loss = 3.9908e-02, PNorm = 40.7373, GNorm = 0.5757, lr_0 = 8.4006e-04
Validation auc = 0.653521
Epoch 110
Validation auc = 0.651174
Epoch 111
Validation auc = 0.659624
Epoch 112
Validation auc = 0.656338
Epoch 113
Validation auc = 0.652113
Epoch 114
Loss = 6.5708e-02, PNorm = 40.8580, GNorm = 0.8338, lr_0 = 8.3288e-04
Validation auc = 0.656808
Epoch 115
Validation auc = 0.660094
Epoch 116
Validation auc = 0.655869
Epoch 117
Validation auc = 0.650235
Epoch 118
Validation auc = 0.658685
Epoch 119
Loss = 5.8268e-02, PNorm = 40.9567, GNorm = 1.6281, lr_0 = 8.2576e-04
Validation auc = 0.661033
Epoch 120
Validation auc = 0.654930
Epoch 121
Validation auc = 0.649765
Epoch 122
Validation auc = 0.656338
Epoch 123
Validation auc = 0.666667
Epoch 124
Loss = 3.7505e-02, PNorm = 41.0567, GNorm = 1.0074, lr_0 = 8.1870e-04
Validation auc = 0.667136
Epoch 125
Validation auc = 0.658685
Epoch 126
Validation auc = 0.655869
Epoch 127
Validation auc = 0.661033
Epoch 128
Validation auc = 0.662441
Epoch 129
Loss = 3.6698e-02, PNorm = 41.1614, GNorm = 1.1149, lr_0 = 8.1170e-04
Validation auc = 0.658685
Epoch 130
Validation auc = 0.656338
Epoch 131
Validation auc = 0.657277
Epoch 132
Validation auc = 0.654930
Epoch 133
Validation auc = 0.657746
Epoch 134
Loss = 1.8575e-02, PNorm = 41.2603, GNorm = 0.5143, lr_0 = 8.0476e-04
Validation auc = 0.656808
Epoch 135
Validation auc = 0.655399
Epoch 136
Validation auc = 0.655399
Epoch 137
Validation auc = 0.659155
Epoch 138
Validation auc = 0.662911
Epoch 139
Loss = 1.1667e-02, PNorm = 41.3421, GNorm = 0.2064, lr_0 = 7.9788e-04
Validation auc = 0.667136
Epoch 140
Validation auc = 0.668075
Epoch 141
Validation auc = 0.671362
Epoch 142
Validation auc = 0.672770
Epoch 143
Validation auc = 0.671831
Epoch 144
Loss = 1.0307e-02, PNorm = 41.4046, GNorm = 0.2035, lr_0 = 7.9106e-04
Validation auc = 0.673239
Epoch 145
Validation auc = 0.670423
Epoch 146
Validation auc = 0.670423
Epoch 147
Validation auc = 0.668545
Epoch 148
Validation auc = 0.664319
Epoch 149
Loss = 9.3931e-03, PNorm = 41.4652, GNorm = 0.1495, lr_0 = 7.8430e-04
Validation auc = 0.662441
Epoch 150
Validation auc = 0.660563
Epoch 151
Validation auc = 0.661502
Epoch 152
Validation auc = 0.668075
Epoch 153
Validation auc = 0.660563
Epoch 154
Loss = 1.9236e-02, PNorm = 41.5342, GNorm = 1.0476, lr_0 = 7.7759e-04
Validation auc = 0.646948
Epoch 155
Validation auc = 0.655399
Epoch 156
Validation auc = 0.653521
Epoch 157
Validation auc = 0.660563
Epoch 158
Validation auc = 0.658216
Epoch 159
Loss = 1.0653e-01, PNorm = 41.6565, GNorm = 2.7398, lr_0 = 7.7095e-04
Validation auc = 0.665258
Epoch 160
Validation auc = 0.662441
Epoch 161
Validation auc = 0.664319
Epoch 162
Validation auc = 0.646479
Epoch 163
Validation auc = 0.646479
Epoch 164
Loss = 3.3244e-02, PNorm = 41.8098, GNorm = 0.4527, lr_0 = 7.6436e-04
Validation auc = 0.646009
Epoch 165
Validation auc = 0.658685
Epoch 166
Validation auc = 0.656338
Epoch 167
Validation auc = 0.658216
Epoch 168
Validation auc = 0.659155
Epoch 169
Loss = 3.5482e-02, PNorm = 41.9321, GNorm = 2.3122, lr_0 = 7.5782e-04
Validation auc = 0.661972
Epoch 170
Validation auc = 0.666197
Epoch 171
Validation auc = 0.669014
Epoch 172
Validation auc = 0.667136
Epoch 173
Validation auc = 0.667606
Epoch 174
Loss = 2.3822e-02, PNorm = 42.0352, GNorm = 0.2576, lr_0 = 7.5134e-04
Validation auc = 0.672300
Epoch 175
Validation auc = 0.676995
Epoch 176
Validation auc = 0.681221
Epoch 177
Validation auc = 0.680751
Epoch 178
Validation auc = 0.680751
Epoch 179
Loss = 1.7472e-02, PNorm = 42.1070, GNorm = 0.1783, lr_0 = 7.4492e-04
Validation auc = 0.678404
Epoch 180
Validation auc = 0.675587
Epoch 181
Validation auc = 0.678404
Epoch 182
Validation auc = 0.684507
Epoch 183
Validation auc = 0.685915
Epoch 184
Loss = 5.8955e-03, PNorm = 42.1741, GNorm = 0.1693, lr_0 = 7.3855e-04
Validation auc = 0.684038
Epoch 185
Validation auc = 0.682160
Epoch 186
Validation auc = 0.681221
Epoch 187
Validation auc = 0.684038
Epoch 188
Validation auc = 0.683568
Epoch 189
Loss = 1.1915e-02, PNorm = 42.2303, GNorm = 0.3848, lr_0 = 7.3224e-04
Validation auc = 0.676526
Epoch 190
Validation auc = 0.665258
Epoch 191
Validation auc = 0.659155
Epoch 192
Validation auc = 0.656808
Epoch 193
Validation auc = 0.660094
Epoch 194
Loss = 7.1593e-03, PNorm = 42.2802, GNorm = 0.0851, lr_0 = 7.2598e-04
Validation auc = 0.664789
Epoch 195
Validation auc = 0.666197
Epoch 196
Validation auc = 0.666667
Epoch 197
Validation auc = 0.666197
Epoch 198
Validation auc = 0.664789
Epoch 199
Loss = 1.1584e-02, PNorm = 42.3235, GNorm = 0.2448, lr_0 = 7.1977e-04
Validation auc = 0.673239
Epoch 200
Validation auc = 0.683099
Epoch 201
Validation auc = 0.688732
Epoch 202
Validation auc = 0.686854
Epoch 203
Validation auc = 0.684977
Epoch 204
Loss = 5.3376e-03, PNorm = 42.3623, GNorm = 0.2054, lr_0 = 7.1362e-04
Validation auc = 0.679343
Epoch 205
Validation auc = 0.675117
Epoch 206
Validation auc = 0.683099
Epoch 207
Validation auc = 0.690610
Epoch 208
Validation auc = 0.692958
Epoch 209
Loss = 5.0159e-03, PNorm = 42.3914, GNorm = 0.1968, lr_0 = 7.0752e-04
Validation auc = 0.692019
Epoch 210
Validation auc = 0.688263
Epoch 211
Validation auc = 0.684977
Epoch 212
Validation auc = 0.680282
Epoch 213
Validation auc = 0.684507
Epoch 214
Loss = 5.8002e-03, PNorm = 42.4162, GNorm = 0.2030, lr_0 = 7.0147e-04
Validation auc = 0.685446
Epoch 215
Validation auc = 0.689671
Epoch 216
Validation auc = 0.696244
Epoch 217
Validation auc = 0.693427
Epoch 218
Validation auc = 0.684977
Epoch 219
Loss = 3.3140e-03, PNorm = 42.4514, GNorm = 0.0455, lr_0 = 6.9548e-04
Validation auc = 0.680751
Epoch 220
Validation auc = 0.678873
Epoch 221
Validation auc = 0.678404
Epoch 222
Validation auc = 0.681221
Epoch 223
Validation auc = 0.683099
Epoch 224
Loss = 8.4637e-03, PNorm = 42.4849, GNorm = 0.3307, lr_0 = 6.8953e-04
Validation auc = 0.682160
Epoch 225
Validation auc = 0.678404
Epoch 226
Validation auc = 0.672770
Epoch 227
Validation auc = 0.669014
Epoch 228
Validation auc = 0.667136
Epoch 229
Loss = 8.6619e-03, PNorm = 42.5247, GNorm = 0.4271, lr_0 = 6.8364e-04
Validation auc = 0.672300
Epoch 230
Validation auc = 0.677934
Epoch 231
Validation auc = 0.679812
Epoch 232
Validation auc = 0.680751
Epoch 233
Validation auc = 0.680282
Epoch 234
Loss = 5.4268e-03, PNorm = 42.5540, GNorm = 0.1450, lr_0 = 6.7779e-04
Validation auc = 0.678873
Epoch 235
Validation auc = 0.671831
Epoch 236
Validation auc = 0.668075
Epoch 237
Validation auc = 0.672770
Epoch 238
Validation auc = 0.677934
Epoch 239
Loss = 4.6564e-03, PNorm = 42.5857, GNorm = 0.1070, lr_0 = 6.7200e-04
Validation auc = 0.677934
Epoch 240
Validation auc = 0.678404
Epoch 241
Validation auc = 0.676995
Epoch 242
Validation auc = 0.673239
Epoch 243
Validation auc = 0.672770
Epoch 244
Loss = 6.8913e-03, PNorm = 42.6201, GNorm = 0.1035, lr_0 = 6.6625e-04
Validation auc = 0.677934
Epoch 245
Validation auc = 0.680282
Epoch 246
Validation auc = 0.681221
Epoch 247
Validation auc = 0.683099
Epoch 248
Validation auc = 0.680751
Epoch 249
Loss = 4.6756e-03, PNorm = 42.6474, GNorm = 0.2987, lr_0 = 6.6056e-04
Validation auc = 0.679343
Epoch 250
Validation auc = 0.677934
Epoch 251
Validation auc = 0.676056
Epoch 252
Validation auc = 0.678404
Epoch 253
Validation auc = 0.679812
Epoch 254
Loss = 3.1705e-03, PNorm = 42.6716, GNorm = 0.0611, lr_0 = 6.5491e-04
Validation auc = 0.684038
Epoch 255
Validation auc = 0.684038
Epoch 256
Validation auc = 0.684507
Epoch 257
Validation auc = 0.684038
Epoch 258
Validation auc = 0.682160
Epoch 259
Loss = 1.9418e-03, PNorm = 42.6934, GNorm = 0.0913, lr_0 = 6.4931e-04
Validation auc = 0.680751
Epoch 260
Validation auc = 0.680751
Epoch 261
Validation auc = 0.682160
Epoch 262
Validation auc = 0.682629
Epoch 263
Validation auc = 0.683099
Epoch 264
Loss = 2.4245e-03, PNorm = 42.7131, GNorm = 0.0498, lr_0 = 6.4376e-04
Validation auc = 0.683099
Epoch 265
Validation auc = 0.683568
Epoch 266
Validation auc = 0.683099
Epoch 267
Validation auc = 0.678873
Epoch 268
Validation auc = 0.677934
Epoch 269
Loss = 3.1194e-03, PNorm = 42.7335, GNorm = 0.1282, lr_0 = 6.3826e-04
Validation auc = 0.675587
Epoch 270
Validation auc = 0.674648
Epoch 271
Validation auc = 0.677934
Epoch 272
Validation auc = 0.681690
Epoch 273
Validation auc = 0.684507
Epoch 274
Loss = 2.0026e-03, PNorm = 42.7575, GNorm = 0.0519, lr_0 = 6.3280e-04
Validation auc = 0.685915
Epoch 275
Validation auc = 0.684977
Epoch 276
Validation auc = 0.681690
Epoch 277
Validation auc = 0.682629
Epoch 278
Validation auc = 0.681221
Epoch 279
Loss = 1.6864e-03, PNorm = 42.7820, GNorm = 0.1024, lr_0 = 6.2739e-04
Validation auc = 0.682160
Epoch 280
Validation auc = 0.684507
Epoch 281
Validation auc = 0.689202
Epoch 282
Validation auc = 0.691080
Epoch 283
Validation auc = 0.688732
Epoch 284
Loss = 2.2193e-03, PNorm = 42.8028, GNorm = 0.0316, lr_0 = 6.2203e-04
Validation auc = 0.686854
Epoch 285
Validation auc = 0.683568
Epoch 286
Validation auc = 0.683568
Epoch 287
Validation auc = 0.682160
Epoch 288
Validation auc = 0.682160
Epoch 289
Loss = 2.6300e-03, PNorm = 42.8217, GNorm = 0.1066, lr_0 = 6.1671e-04
Validation auc = 0.682160
Epoch 290
Validation auc = 0.683099
Epoch 291
Validation auc = 0.683099
Epoch 292
Validation auc = 0.685446
Epoch 293
Validation auc = 0.684507
Epoch 294
Loss = 2.1766e-03, PNorm = 42.8388, GNorm = 0.0783, lr_0 = 6.1144e-04
Validation auc = 0.683099
Epoch 295
Validation auc = 0.679343
Epoch 296
Validation auc = 0.677934
Epoch 297
Validation auc = 0.678873
Epoch 298
Validation auc = 0.677934
Epoch 299
Loss = 1.1277e-03, PNorm = 42.8541, GNorm = 0.0431, lr_0 = 6.0621e-04
Validation auc = 0.676995
Model 0 best validation auc = 0.746948 on epoch 7
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test auc = 0.697772
Ensemble test auc = 0.697772
Fold 9
Splitting data with seed 9
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.05, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=300, bias=False)
        (W_h): Linear(in_features=300, out_features=300, bias=False)
        (W_o): Linear(in_features=433, out_features=300, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.05, inplace=False)
    (1): Linear(in_features=2200, out_features=300, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.05, inplace=False)
    (4): Linear(in_features=300, out_features=1, bias=True)
  )
)
Number of parameters = 924,901
Moving model to cuda
Epoch 0
Validation auc = 0.667136
Epoch 1
Validation auc = 0.707981
Epoch 2
Validation auc = 0.683568
Epoch 3
Validation auc = 0.705164
Epoch 4
Loss = 7.2451e-01, PNorm = 37.1925, GNorm = 3.7341, lr_0 = 6.5000e-04
Validation auc = 0.719718
Epoch 5
Validation auc = 0.685915
Epoch 6
Validation auc = 0.677934
Epoch 7
Validation auc = 0.637559
Epoch 8
Validation auc = 0.544131
Epoch 9
Loss = 6.6921e-01, PNorm = 37.2517, GNorm = 15.7731, lr_0 = 9.9743e-04
Validation auc = 0.634742
Epoch 10
Validation auc = 0.636620
Epoch 11
Validation auc = 0.653521
Epoch 12
Validation auc = 0.633803
Epoch 13
Validation auc = 0.640376
Epoch 14
Loss = 5.2019e-01, PNorm = 37.3566, GNorm = 6.9473, lr_0 = 9.8890e-04
Validation auc = 0.637089
Epoch 15
Validation auc = 0.631925
Epoch 16
Validation auc = 0.628638
Epoch 17
Validation auc = 0.625352
Epoch 18
Validation auc = 0.601408
Epoch 19
Loss = 4.1016e-01, PNorm = 37.4702, GNorm = 7.1960, lr_0 = 9.8045e-04
Validation auc = 0.585915
Epoch 20
Validation auc = 0.613146
Epoch 21
Validation auc = 0.594366
Epoch 22
Validation auc = 0.610329
Epoch 23
Validation auc = 0.629577
Epoch 24
Loss = 3.5587e-01, PNorm = 37.6059, GNorm = 7.4275, lr_0 = 9.7207e-04
Validation auc = 0.634742
Epoch 25
Validation auc = 0.611268
Epoch 26
Validation auc = 0.622066
Epoch 27
Validation auc = 0.618779
Epoch 28
Validation auc = 0.581690
Epoch 29
Loss = 4.2827e-01, PNorm = 37.7465, GNorm = 9.7623, lr_0 = 9.6376e-04
Validation auc = 0.611268
Epoch 30
Validation auc = 0.619718
Epoch 31
Validation auc = 0.608451
Epoch 32
Validation auc = 0.574648
Epoch 33
Validation auc = 0.620188
Epoch 34
Loss = 3.4469e-01, PNorm = 37.9039, GNorm = 4.2262, lr_0 = 9.5552e-04
Validation auc = 0.627230
Epoch 35
Validation auc = 0.615023
Epoch 36
Validation auc = 0.620657
Epoch 37
Validation auc = 0.618310
Epoch 38
Validation auc = 0.607042
Epoch 39
Loss = 3.3026e-01, PNorm = 38.0678, GNorm = 1.3762, lr_0 = 9.4735e-04
Validation auc = 0.603756
Epoch 40
Validation auc = 0.602347
Epoch 41
Validation auc = 0.588732
Epoch 42
Validation auc = 0.612207
Epoch 43
Validation auc = 0.622066
Epoch 44
Loss = 2.9095e-01, PNorm = 38.2655, GNorm = 2.5778, lr_0 = 9.3925e-04
Validation auc = 0.599061
Epoch 45
Validation auc = 0.596244
Epoch 46
Validation auc = 0.625352
Epoch 47
Validation auc = 0.615962
Epoch 48
Validation auc = 0.630986
Epoch 49
Loss = 3.0984e-01, PNorm = 38.4866, GNorm = 6.8404, lr_0 = 9.3122e-04
Validation auc = 0.627700
Epoch 50
Validation auc = 0.617840
Epoch 51
Validation auc = 0.611737
Epoch 52
Validation auc = 0.597183
Epoch 53
Validation auc = 0.600469
Epoch 54
Loss = 2.2636e-01, PNorm = 38.6805, GNorm = 5.2193, lr_0 = 9.2326e-04
Validation auc = 0.614554
Epoch 55
Validation auc = 0.613146
Epoch 56
Validation auc = 0.589671
Epoch 57
Validation auc = 0.630047
Epoch 58
Validation auc = 0.631925
Epoch 59
Loss = 1.7354e-01, PNorm = 38.8926, GNorm = 0.6220, lr_0 = 9.1537e-04
Validation auc = 0.605164
Epoch 60
Validation auc = 0.614554
Epoch 61
Validation auc = 0.620188
Epoch 62
Validation auc = 0.606573
Epoch 63
Validation auc = 0.588263
Epoch 64
Loss = 1.7866e-01, PNorm = 39.0867, GNorm = 2.3618, lr_0 = 9.0754e-04
Validation auc = 0.600000
Epoch 65
Validation auc = 0.619718
Epoch 66
Validation auc = 0.607981
Epoch 67
Validation auc = 0.616432
Epoch 68
Validation auc = 0.613615
Epoch 69
Loss = 8.8316e-02, PNorm = 39.2847, GNorm = 0.8686, lr_0 = 8.9978e-04
Validation auc = 0.617371
Epoch 70
Validation auc = 0.622066
Epoch 71
Validation auc = 0.639906
Epoch 72
Validation auc = 0.635681
Epoch 73
Validation auc = 0.647887
Epoch 74
Loss = 1.0346e-01, PNorm = 39.4473, GNorm = 1.4653, lr_0 = 8.9209e-04
Validation auc = 0.647418
Epoch 75
Validation auc = 0.645070
Epoch 76
Validation auc = 0.642254
Epoch 77
Validation auc = 0.618779
Epoch 78
Validation auc = 0.621127
Epoch 79
Loss = 9.3267e-02, PNorm = 39.5997, GNorm = 1.0261, lr_0 = 8.8447e-04
Validation auc = 0.616432
Epoch 80
Validation auc = 0.615962
Epoch 81
Validation auc = 0.617371
Epoch 82
Validation auc = 0.632864
Epoch 83
Validation auc = 0.630986
Epoch 84
Loss = 8.5453e-02, PNorm = 39.7671, GNorm = 0.8252, lr_0 = 8.7691e-04
Validation auc = 0.621127
Epoch 85
Validation auc = 0.618779
Epoch 86
Validation auc = 0.633333
Epoch 87
Validation auc = 0.642254
Epoch 88
Validation auc = 0.641315
Epoch 89
Loss = 7.3806e-02, PNorm = 39.9356, GNorm = 0.4865, lr_0 = 8.6941e-04
Validation auc = 0.647887
Epoch 90
Validation auc = 0.653052
Epoch 91
Validation auc = 0.645070
Epoch 92
Validation auc = 0.632864
Epoch 93
Validation auc = 0.644131
Epoch 94
Loss = 4.7465e-02, PNorm = 40.0930, GNorm = 1.0996, lr_0 = 8.6198e-04
Validation auc = 0.643662
Epoch 95
Validation auc = 0.641784
Epoch 96
Validation auc = 0.639437
Epoch 97
Validation auc = 0.655399
Epoch 98
Validation auc = 0.654930
Epoch 99
Loss = 5.0533e-02, PNorm = 40.2331, GNorm = 1.5461, lr_0 = 8.5461e-04
Validation auc = 0.650235
Epoch 100
Validation auc = 0.664319
Epoch 101
Validation auc = 0.645540
Epoch 102
Validation auc = 0.631455
Epoch 103
Validation auc = 0.616432
Epoch 104
Loss = 6.6950e-02, PNorm = 40.3727, GNorm = 1.0877, lr_0 = 8.4730e-04
Validation auc = 0.614554
Epoch 105
Validation auc = 0.615962
Epoch 106
Validation auc = 0.630516
Epoch 107
Validation auc = 0.639437
Epoch 108
Validation auc = 0.644131
Epoch 109
Loss = 5.3287e-02, PNorm = 40.5318, GNorm = 0.7965, lr_0 = 8.4006e-04
Validation auc = 0.669484
Epoch 110
Validation auc = 0.652113
Epoch 111
Validation auc = 0.633803
Epoch 112
Validation auc = 0.635681
Epoch 113
Validation auc = 0.632394
Epoch 114
Loss = 5.5750e-02, PNorm = 40.6662, GNorm = 1.1075, lr_0 = 8.3288e-04
Validation auc = 0.630986
Epoch 115
Validation auc = 0.630986
Epoch 116
Validation auc = 0.635211
Epoch 117
Validation auc = 0.639906
Epoch 118
Validation auc = 0.632864
Epoch 119
Loss = 3.3338e-02, PNorm = 40.8181, GNorm = 1.2428, lr_0 = 8.2576e-04
Validation auc = 0.638498
Epoch 120
Validation auc = 0.645070
Epoch 121
Validation auc = 0.642723
Epoch 122
Validation auc = 0.646479
Epoch 123
Validation auc = 0.653052
Epoch 124
Loss = 1.8484e-02, PNorm = 40.9441, GNorm = 0.6312, lr_0 = 8.1870e-04
Validation auc = 0.654930
Epoch 125
Validation auc = 0.652113
Epoch 126
Validation auc = 0.664789
Epoch 127
Validation auc = 0.669014
Epoch 128
Validation auc = 0.669484
Epoch 129
Loss = 9.5541e-03, PNorm = 41.0376, GNorm = 0.2730, lr_0 = 8.1170e-04
Validation auc = 0.666667
Epoch 130
Validation auc = 0.673709
Epoch 131
Validation auc = 0.677465
Epoch 132
Validation auc = 0.664319
Epoch 133
Validation auc = 0.646479
Epoch 134
Loss = 3.1011e-02, PNorm = 41.1267, GNorm = 2.3836, lr_0 = 8.0476e-04
Validation auc = 0.643662
Epoch 135
Validation auc = 0.651174
Epoch 136
Validation auc = 0.654460
Epoch 137
Validation auc = 0.649765
Epoch 138
Validation auc = 0.655399
Epoch 139
Loss = 4.8670e-02, PNorm = 41.2162, GNorm = 0.6947, lr_0 = 7.9788e-04
Validation auc = 0.672770
Epoch 140
Validation auc = 0.669953
Epoch 141
Validation auc = 0.661033
Epoch 142
Validation auc = 0.650704
Epoch 143
Validation auc = 0.652582
Epoch 144
Loss = 3.9744e-02, PNorm = 41.3074, GNorm = 0.5641, lr_0 = 7.9106e-04
Validation auc = 0.655399
Epoch 145
Validation auc = 0.653052
Epoch 146
Validation auc = 0.651174
Epoch 147
Validation auc = 0.650704
Epoch 148
Validation auc = 0.657746
Epoch 149
Loss = 5.4196e-02, PNorm = 41.4090, GNorm = 2.6154, lr_0 = 7.8430e-04
Validation auc = 0.658685
Epoch 150
Validation auc = 0.649296
Epoch 151
Validation auc = 0.637089
Epoch 152
Validation auc = 0.634272
Epoch 153
Validation auc = 0.634742
Epoch 154
Loss = 2.9103e-02, PNorm = 41.5195, GNorm = 0.7263, lr_0 = 7.7759e-04
Validation auc = 0.642254
Epoch 155
Validation auc = 0.638028
Epoch 156
Validation auc = 0.643192
Epoch 157
Validation auc = 0.647418
Epoch 158
Validation auc = 0.670423
Epoch 159
Loss = 7.1769e-02, PNorm = 41.6314, GNorm = 5.4693, lr_0 = 7.7095e-04
Validation auc = 0.660563
Epoch 160
Validation auc = 0.646479
Epoch 161
Validation auc = 0.645070
Epoch 162
Validation auc = 0.656808
Epoch 163
Validation auc = 0.654930
Epoch 164
Loss = 4.8782e-02, PNorm = 41.7440, GNorm = 1.4091, lr_0 = 7.6436e-04
Validation auc = 0.653991
Epoch 165
Validation auc = 0.640845
Epoch 166
Validation auc = 0.646009
Epoch 167
Validation auc = 0.654460
Epoch 168
Validation auc = 0.658685
Epoch 169
Loss = 3.7328e-02, PNorm = 41.8577, GNorm = 0.4129, lr_0 = 7.5782e-04
Validation auc = 0.657746
Epoch 170
Validation auc = 0.651174
Epoch 171
Validation auc = 0.652113
Epoch 172
Validation auc = 0.655869
Epoch 173
Validation auc = 0.657746
Epoch 174
Loss = 1.4484e-02, PNorm = 41.9498, GNorm = 0.5465, lr_0 = 7.5134e-04
Validation auc = 0.660094
Epoch 175
Validation auc = 0.658216
Epoch 176
Validation auc = 0.661502
Epoch 177
Validation auc = 0.660094
Epoch 178
Validation auc = 0.657746
Epoch 179
Loss = 1.2462e-02, PNorm = 42.0213, GNorm = 0.5083, lr_0 = 7.4492e-04
Validation auc = 0.657277
Epoch 180
Validation auc = 0.661502
Epoch 181
Validation auc = 0.658685
Epoch 182
Validation auc = 0.657277
Epoch 183
Validation auc = 0.653052
Epoch 184
Loss = 3.9373e-02, PNorm = 42.0805, GNorm = 1.6428, lr_0 = 7.3855e-04
Validation auc = 0.660563
Epoch 185
Validation auc = 0.666197
Epoch 186
Validation auc = 0.661033
Epoch 187
Validation auc = 0.656808
Epoch 188
Validation auc = 0.656338
Epoch 189
Loss = 1.0732e-02, PNorm = 42.1437, GNorm = 0.2192, lr_0 = 7.3224e-04
Validation auc = 0.656808
Epoch 190
Validation auc = 0.656808
Epoch 191
Validation auc = 0.658216
Epoch 192
Validation auc = 0.659155
Epoch 193
Validation auc = 0.659155
Epoch 194
Loss = 9.5533e-03, PNorm = 42.1902, GNorm = 0.3102, lr_0 = 7.2598e-04
Validation auc = 0.654930
Epoch 195
Validation auc = 0.651643
Epoch 196
Validation auc = 0.649296
Epoch 197
Validation auc = 0.655869
Epoch 198
Validation auc = 0.659155
Epoch 199
Loss = 2.9617e-03, PNorm = 42.2392, GNorm = 0.0869, lr_0 = 7.1977e-04
Validation auc = 0.661033
Epoch 200
Validation auc = 0.668075
Epoch 201
Validation auc = 0.673709
Epoch 202
Validation auc = 0.676526
Epoch 203
Validation auc = 0.678404
Epoch 204
Loss = 6.3160e-03, PNorm = 42.2727, GNorm = 0.0697, lr_0 = 7.1362e-04
Validation auc = 0.680282
Epoch 205
Validation auc = 0.680751
Epoch 206
Validation auc = 0.680282
Epoch 207
Validation auc = 0.678873
Epoch 208
Validation auc = 0.681690
Epoch 209
Loss = 5.5936e-03, PNorm = 42.2960, GNorm = 0.2732, lr_0 = 7.0752e-04
Validation auc = 0.681690
Epoch 210
Validation auc = 0.677465
Epoch 211
Validation auc = 0.669484
Epoch 212
Validation auc = 0.665728
Epoch 213
Validation auc = 0.663380
Epoch 214
Loss = 6.5942e-03, PNorm = 42.3393, GNorm = 0.0389, lr_0 = 7.0147e-04
Validation auc = 0.665728
Epoch 215
Validation auc = 0.667136
Epoch 216
Validation auc = 0.668075
Epoch 217
Validation auc = 0.668075
Epoch 218
Validation auc = 0.666667
Epoch 219
Loss = 1.8408e-03, PNorm = 42.3785, GNorm = 0.0203, lr_0 = 6.9548e-04
Validation auc = 0.665258
Epoch 220
Validation auc = 0.664789
Epoch 221
Validation auc = 0.665728
Epoch 222
Validation auc = 0.668545
Epoch 223
Validation auc = 0.675117
Epoch 224
Loss = 9.9122e-03, PNorm = 42.4059, GNorm = 0.1848, lr_0 = 6.8953e-04
Validation auc = 0.678873
Epoch 225
Validation auc = 0.677934
Epoch 226
Validation auc = 0.674178
Epoch 227
Validation auc = 0.667136
Epoch 228
Validation auc = 0.662911
Epoch 229
Loss = 4.2344e-03, PNorm = 42.4410, GNorm = 0.3783, lr_0 = 6.8364e-04
Validation auc = 0.661033
Epoch 230
Validation auc = 0.661972
Epoch 231
Validation auc = 0.663850
Epoch 232
Validation auc = 0.665258
Epoch 233
Validation auc = 0.670423
Epoch 234
Loss = 4.0937e-03, PNorm = 42.4684, GNorm = 0.1949, lr_0 = 6.7779e-04
Validation auc = 0.669484
Epoch 235
Validation auc = 0.668075
Epoch 236
Validation auc = 0.663850
Epoch 237
Validation auc = 0.661972
Epoch 238
Validation auc = 0.659624
Epoch 239
Loss = 9.0552e-03, PNorm = 42.4942, GNorm = 0.7629, lr_0 = 6.7200e-04
Validation auc = 0.662441
Epoch 240
Validation auc = 0.668545
Epoch 241
Validation auc = 0.675117
Epoch 242
Validation auc = 0.678873
Epoch 243
Validation auc = 0.682629
Epoch 244
Loss = 2.5600e-03, PNorm = 42.5173, GNorm = 0.1022, lr_0 = 6.6625e-04
Validation auc = 0.682629
Epoch 245
Validation auc = 0.680282
Epoch 246
Validation auc = 0.675587
Epoch 247
Validation auc = 0.675117
Epoch 248
Validation auc = 0.676526
Epoch 249
Loss = 4.8489e-03, PNorm = 42.5441, GNorm = 0.0580, lr_0 = 6.6056e-04
Validation auc = 0.679812
Epoch 250
Validation auc = 0.684977
Epoch 251
Validation auc = 0.687324
Epoch 252
Validation auc = 0.683099
Epoch 253
Validation auc = 0.678873
Epoch 254
Loss = 1.8929e-03, PNorm = 42.5712, GNorm = 0.0166, lr_0 = 6.5491e-04
Validation auc = 0.676056
Epoch 255
Validation auc = 0.675587
Epoch 256
Validation auc = 0.676056
Epoch 257
Validation auc = 0.681690
Epoch 258
Validation auc = 0.682629
Epoch 259
Loss = 3.1042e-03, PNorm = 42.5961, GNorm = 0.2152, lr_0 = 6.4931e-04
Validation auc = 0.677934
Epoch 260
Validation auc = 0.674178
Epoch 261
Validation auc = 0.670423
Epoch 262
Validation auc = 0.669484
Epoch 263
Validation auc = 0.667136
Epoch 264
Loss = 2.3906e-03, PNorm = 42.6241, GNorm = 0.0116, lr_0 = 6.4376e-04
Validation auc = 0.666667
Epoch 265
Validation auc = 0.666197
Epoch 266
Validation auc = 0.665728
Epoch 267
Validation auc = 0.667606
Epoch 268
Validation auc = 0.669484
Epoch 269
Loss = 1.6079e-03, PNorm = 42.6431, GNorm = 0.0454, lr_0 = 6.3826e-04
Validation auc = 0.669953
Epoch 270
Validation auc = 0.669484
Epoch 271
Validation auc = 0.668545
Epoch 272
Validation auc = 0.669014
Epoch 273
Validation auc = 0.668075
Epoch 274
Loss = 1.8186e-03, PNorm = 42.6610, GNorm = 0.0836, lr_0 = 6.3280e-04
Validation auc = 0.667606
Epoch 275
Validation auc = 0.670423
Epoch 276
Validation auc = 0.673709
Epoch 277
Validation auc = 0.676526
Epoch 278
Validation auc = 0.676995
Epoch 279
Loss = 1.4559e-03, PNorm = 42.6768, GNorm = 0.0282, lr_0 = 6.2739e-04
Validation auc = 0.675587
Epoch 280
Validation auc = 0.675587
Epoch 281
Validation auc = 0.675117
Epoch 282
Validation auc = 0.672770
Epoch 283
Validation auc = 0.672300
Epoch 284
Loss = 1.4079e-03, PNorm = 42.6931, GNorm = 0.0118, lr_0 = 6.2203e-04
Validation auc = 0.675587
Epoch 285
Validation auc = 0.676526
Epoch 286
Validation auc = 0.675587
Epoch 287
Validation auc = 0.676995
Epoch 288
Validation auc = 0.678873
Epoch 289
Loss = 1.5141e-03, PNorm = 42.7074, GNorm = 0.0341, lr_0 = 6.1671e-04
Validation auc = 0.678404
Epoch 290
Validation auc = 0.677934
Epoch 291
Validation auc = 0.677934
Epoch 292
Validation auc = 0.677934
Epoch 293
Validation auc = 0.677465
Epoch 294
Loss = 1.7952e-03, PNorm = 42.7200, GNorm = 0.0433, lr_0 = 6.1144e-04
Validation auc = 0.679343
Epoch 295
Validation auc = 0.680751
Epoch 296
Validation auc = 0.680751
Epoch 297
Validation auc = 0.681221
Epoch 298
Validation auc = 0.680751
Epoch 299
Loss = 9.2417e-04, PNorm = 42.7323, GNorm = 0.0270, lr_0 = 6.0621e-04
Validation auc = 0.679812
Model 0 best validation auc = 0.719718 on epoch 4
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test auc = 0.742926
Ensemble test auc = 0.742926
10-fold cross validation
	Seed 0 ==> test auc = 0.747140
	Seed 1 ==> test auc = 0.720650
	Seed 2 ==> test auc = 0.783865
	Seed 3 ==> test auc = 0.738712
	Seed 4 ==> test auc = 0.745334
	Seed 5 ==> test auc = 0.721252
	Seed 6 ==> test auc = 0.754365
	Seed 7 ==> test auc = 0.762191
	Seed 8 ==> test auc = 0.697772
	Seed 9 ==> test auc = 0.742926
Overall test auc = 0.741421 +/- 0.022807
Elapsed time = 0:06:54
